Bubble Entropy: An Entropy Almost Free of Parameters

EEG-Based Mental Workload Classification Method Based on Hybrid Deep Learning Model Under IoT

Automatically detecting human mental workload to prevent mental diseases is highly important. With the development of information technology, remote detection of mental workload is expected. The development of artificial intelligence and Internet of Things technology will also enable the identification of mental workload remotely based on human physiological signals. In this article, a method based on the spatial and time-frequency domains of electroencephalography (EEG) signals is proposed to improve the classification accuracy of mental workload. Moreover, a hybrid deep learning model is presented. First, the spatial domain features of different brain regions are proposed. Simultaneously, EEG time-frequency domain information is obtained based on wavelet transform. The spatial and time-frequency domain features are input into two types of deep learning models for mental workload classification. To validate the performance of the proposed method, the Simultaneous Task EEG Workload public database is used. Compared with the existing methods, the proposed approach shows higher classification accuracy. It provides a novel means of assessing mental workload.

Remote passive acoustic signal detection using multi-scale correlation networks and network spectrum distance in marine environment

Detecting acoustic signals in the ocean is crucial for port and coastal security, but existing methods often require informative priors. This paper introduces a new approach that transforms acoustic signal detection into network characterization using a MCN construction method. The method constructs a network representation of the acoustic signal by measuring pairwise correlations at different time scales. It proposes a network spectrum distance method that combines information geometry and graph signal processing theory to characterize these complex networks. By comparing the spectra of two networks, the method quantifies their similarity or dissimilarity, enabling comparisons of multi-scale correlation networks constructed from different time series data and tracking changes in nonlinear dynamics over time. The effectiveness of these methods is substantiated through comprehensive simulations and real-world data collected from the South China Sea. The results illustrate that the proposed approach attains a significant detection probability of over 90% when the signal-to-noise ratio exceeds -18 dB, whereas existing methods require a signal-to-noise ratio of at least -15 dB to achieve a comparable detection probability. This innovative approach holds promising applications in bolstering port security, facilitating coastal operations, and optimizing offshore activities by enabling more efficient detection of weak acoustic signals.

Characterization and separation of preterm and term spontaneous, induced, and cesarean EHG records

To improve the understanding of the underlying physiological processes that lead to preterm birth, and different term delivery modes, we quantitatively characterized and assessed the separability of the sets of early (23rd week) and later (31st week) recorded, preterm and term spontaneous, induced, cesarean, and induced-cesarean electrohysterogram (EHG) records using several of the most widely used non-linear features extracted from the EHG signals. Linearly modeled temporal trends of the means of the median frequencies (MFs), and of the means of the peak amplitudes (PAs) of the normalized power spectra of the EHG signals, along pregnancy (from early to later recorded records), derived from a variety of frequency bands, revealed that for the preterm group of records, in comparison to all other term delivery groups, the frequency spectrum of the frequency band B0L (0.08-0.3 Hz) shifts toward higher frequencies, and that the spectrum of the newly identified frequency band B0L' (0.125-0.575 Hz), which approximately matches the Fast Wave Low band, becomes stronger. The most promising features to separate between the later preterm group and all other later term delivery groups appear to be MF (p=1.1⋅10^-5) in the band B0L of the horizontal signal S3, and PA (p=2.4⋅10^-8) in the band B0L' (S3). Moreover, the PA in the band B0L' (S3) showed the highest power to individually separate between the later preterm group and any other later term delivery group. Furthermore, the results suggest that in preterm pregnancies the resting maternal heart rate decreases between the 23rd and 31st week of gestation.

Slope Entropy Characterisation: The Role of the δ Parameter

Many time series entropy calculation methods have been proposed in the last few years. They are mainly used as numerical features for signal classification in any scientific field where data series are involved. We recently proposed a new method, Slope Entropy (SlpEn), based on the relative frequency of differences between consecutive samples of a time series, thresholded using two input parameters, γ and δ. In principle, δ was proposed to account for differences in the vicinity of the 0 region (namely, ties) and, therefore, was usually set at small values such as 0.001. However, there is no study that really quantifies the role of this parameter using this default or other configurations, despite the good SlpEn results so far. The present paper addresses this issue, removing δ from the SlpEn calculation to assess its real influence on classification performance, or optimising its value by means of a grid search in order to find out if other values beyond the 0.001 value provide significant time series classification accuracy gains. Although the inclusion of this parameter does improve classification accuracy according to experimental results, gains of 5% at most probably do not support the additional effort required. Therefore, SlpEn simplification could be seen as a real alternative.

Calcium Chloride Toxicology for Food Safety Assessment Using Zebrafish (Danio rerio) Embryos

The salt calcium chloride (CaCl₂) is widely used in industry as a food additive; levels for human consumption are regulated by international or governmental agencies. Generally, the food industry relies on toxicity studies conducted in mammals such as mice, rats, and rabbits for determining food safety. However, testing in mammals is time-consuming and expensive. Zebrafish have been used in a range of toxicological analyses and offer advantages with regard to sensitivity, time, and cost. However, information in not available with regard to whether the sensitivity of zebrafish to CaCl₂ is comparable to the concentrations of CaCl₂ used as food additives. The aim of this study was to compare the CaCl₂ tolerance of zebrafish embryos and larvae with concentrations currently approved as food additives. Acute toxicity, embryotoxicity, cardiotoxicity, and neurotoxicity assays were used to determine the threshold toxic concentration of CaCl₂ in zebrafish embryos and larvae. The data showed that doses above 0.4% had toxic effects on development and on the activity of the cardiac and neuronal systems. Furthermore, all embryos exposed to 0.8 and 1.6% of CaCl₂ died after 24 hpf. These findings are consistent with the limits of CaCl₂ concentrations approved by Codex Alimentarius. Therefore, zebrafish embryos could be suitable for screening food additives.

Fault Diagnosis of Power Transformer Based on Time-Shift Multiscale Bubble Entropy and Stochastic Configuration Network

In order to accurately diagnose the fault type of power transformer, this paper proposes a transformer fault diagnosis method based on the combination of time-shift multiscale bubble entropy (TSMBE) and stochastic configuration network (SCN). Firstly, bubble entropy is introduced to overcome the shortcomings of traditional entropy models that rely too heavily on hyperparameters. Secondly, on the basis of bubble entropy, a tool for measuring signal complexity, TSMBE, is proposed. Then, the TSMBE of the transformer vibration signal is extracted as a fault feature. Finally, the fault feature is inputted into the stochastic configuration network model to achieve an accurate identification of different transformer state signals. The proposed method was applied to real power transformer fault cases, and the research results showed that TSMBE-SCN achieved 99.01%, 99.1%, 99.11%, 99.11%, 99.14% and 99.02% of the diagnostic rates under different folding numbers, respectively, compared with conventional diagnostic models MBE-SCN, TSMSE-SCN, MSE-SCN, TSMDE-SCN and MDE-SCN. This comparison shows that TSMBE-SCN has a strong competitive advantage, which verifies that the proposed method has a good diagnostic effect. This study provides a new method for power transformer fault diagnosis, which has good reference value.

Pressure Injury Link to Entropy of Abdominal Temperature

This study examined the association between pressure injuries and complexity of abdominal temperature measured in residents of a nursing facility. The temperature served as a proxy measure for skin thermoregulation. Refined multiscale sample entropy and bubble entropy were used to measure the irregularity of the temperature time series measured over two days at 1-min intervals. Robust summary measures were derived for the multiscale entropies and used in predictive models for pressure injuries that were built with adaptive lasso regression and neural networks. Both types of entropies were lower in the group of participants with pressure injuries (n=11) relative to the group of non-injured participants (n=15). This was generally true at the longer temporal scales, with the effect peaking at scale τ=22 min for sample entropy and τ=23 min for bubble entropy. Predictive models for pressure injury on the basis of refined multiscale sample entropy and bubble entropy yielded 96% accuracy, outperforming predictions based on any single measure of entropy. Combining entropy measures with a widely used risk assessment score led to the best prediction accuracy. Complexity of the abdominal temperature series could therefore serve as an indicator of risk of pressure injury.

The Structure of Chaos: An Empirical Comparison of Fractal Physiology Complexity Indices Using NeuroKit2

Complexity quantification, through entropy, information theory and fractal dimension indices, is gaining a renewed traction in psychophsyiology, as new measures with promising qualities emerge from the computational and mathematical advances. Unfortunately, few studies compare the relationship and objective performance of the plethora of existing metrics, in turn hindering reproducibility, replicability, consistency, and clarity in the field. Using the NeuroKit2 Python software, we computed a list of 112 (predominantly used) complexity indices on signals varying in their characteristics (noise, length and frequency spectrum). We then systematically compared the indices by their computational weight, their representativeness of a multidimensional space of latent dimensions, and empirical proximity with other indices. Based on these considerations, we propose that a selection of 12 indices, together representing 85.97% of the total variance of all indices, might offer a parsimonious and complimentary choice in regards to the quantification of the complexity of time series. Our selection includes CWPEn, Line Length (LL), BubbEn, MSWPEn, MFDFA (Max), Hjorth Complexity, SVDEn, MFDFA (Width), MFDFA (Mean), MFDFA (Peak), MFDFA (Fluctuation), AttEn. Elements of consideration for alternative subsets are discussed, and data, analysis scripts and code for the figures are open-source.

Optimal Classification of Atrial Fibrillation and Congestive Heart Failure Using Machine Learning

Cardiovascular disorders, including atrial fibrillation (AF) and congestive heart failure (CHF), are the significant causes of mortality worldwide. The diagnosis of cardiovascular disorders is heavily reliant on ECG signals. Therefore, extracting significant features from ECG signals is the most challenging aspect of representing each condition of ECG signal. Earlier studies have claimed that the Hjorth descriptor is assigned as a simple feature extraction algorithm capable of class separation among AF, CHF, and normal sinus rhythm (NSR) conditions. However, due to noise interference, certain features do not represent the characteristics of the ECG signals. This study addressed this critical gap by applying the discrete wavelet transform (DWT) to decompose the ECG signals into sub-bands and extracting Hjorth descriptor features and entropy-based features in the DWT domain. Therefore, the calculation of Hjorth descriptor and entropy-based features performed on each sub-band will produce more detailed information of ECG signals. The optimization of various classifier algorithms, including k-nearest neighbor (k-NN), support vector machine (SVM), random forest (RF), artificial neural network (ANN), and radial basis function network (RBFN), was investigated to provide the best system performance. This study obtained an accuracy of 100% for the k-NN, SVM, RF, and ANN classifiers, respectively, and 97% for the RBFN classifier. The results demonstrated that the optimization of the classifier algorithm could improve the classification accuracy of AF, CHF, and NSR conditions, compared to earlier studies.

Enhancing classification of preterm-term birth using continuous wavelet transform and entropy-based methods of electrohysterogram signals

INTRODUCTION

Despite vast research, premature birth's electrophysiological mechanisms are not fully understood. Prediction of preterm birth contributes to child survival by providing timely and skilled care to both mother and child. Electrohysterography is an affordable, noninvasive technique that has been highly sensitive in diagnosing preterm labor. This study aimed to choose the more appropriate combination of characteristics, such as electrode channel and bandwidth, as well as those linear, time-frequency, and nonlinear features of the electrohysterogram (EHG) for predicting preterm birth using classifiers.

METHODS

We analyzed two open-access datasets of 30 minutes of EHG obtained in regular checkups of women around 31 weeks of pregnancy who experienced premature labor (P) and term labor (T). The current approach filtered the raw EHGs in three relevant frequency subbands (0.3-1 Hz, 1-2 Hz, and 2-3Hz). The EHG time series were then segmented to create 120-second windows, from which individual characteristics were calculated. The linear, time-frequency, and nonlinear indices of EHG of each combination (channel-filter) were fed to different classifiers using feature selection techniques.

RESULTS

The best performance, i.e., 88.52% accuracy, 83.83% sensitivity, and 93.22% specificity, was obtained in the 2-3 Hz bands using Medium Frequency, Continuous Wavelet Transform (CWT), and entropy-based indices. Interestingly, CWT features were significantly different in all filter-channel combinations. The proposed study uses small samples of EHG signals to diagnose preterm birth accurately, showing their potential application in the clinical environment.

DISCUSSION

Our results suggest that CWT and novel entropy-based features of EHG could be suitable descriptors for analyzing and understanding the complex nature of preterm labor mechanisms.

EntropyHub: An open-source toolkit for entropic time series analysis

An increasing number of studies across many research fields from biomedical engineering to finance are employing measures of entropy to quantify the regularity, variability or randomness of time series and image data. Entropy, as it relates to information theory and dynamical systems theory, can be estimated in many ways, with newly developed methods being continuously introduced in the scientific literature. Despite the growing interest in entropic time series and image analysis, there is a shortage of validated, open-source software tools that enable researchers to apply these methods. To date, packages for performing entropy analysis are often run using graphical user interfaces, lack the necessary supporting documentation, or do not include functions for more advanced entropy methods, such as cross-entropy, multiscale cross-entropy or bidimensional entropy. In light of this, this paper introduces EntropyHub, an open-source toolkit for performing entropic time series analysis in MATLAB, Python and Julia. EntropyHub (version 0.1) provides an extensive range of more than forty functions for estimating cross-, multiscale, multiscale cross-, and bidimensional entropy, each including a number of keyword arguments that allows the user to specify multiple parameters in the entropy calculation. Instructions for installation, descriptions of function syntax, and examples of use are fully detailed in the supporting documentation, available on the EntropyHub website- www.EntropyHub.xyz. Compatible with Windows, Mac and Linux operating systems, EntropyHub is hosted on GitHub, as well as the native package repository for MATLAB, Python and Julia, respectively. The goal of EntropyHub is to integrate the many established entropy methods into one complete resource, providing tools that make advanced entropic time series analysis straightforward and reproducible.

Assessment of Dispersion and Bubble Entropy Measures for Enhancing Preterm Birth Prediction Based on Electrohysterographic Signals

One of the remaining challenges for the scientific-technical community is predicting preterm births, for which electrohysterography (EHG) has emerged as a highly sensitive prediction technique. Sample and fuzzy entropy have been used to characterize EHG signals, although they require optimizing many internal parameters. Both bubble entropy, which only requires one internal parameter, and dispersion entropy, which can detect any changes in frequency and amplitude, have been proposed to characterize biomedical signals. In this work, we attempted to determine the clinical value of these entropy measures for predicting preterm birth by analyzing their discriminatory capacity as an individual feature and their complementarity to other EHG characteristics by developing six prediction models using obstetrical data, linear and non-linear EHG features, and linear discriminant analysis using a genetic algorithm to select the features. Both dispersion and bubble entropy better discriminated between the preterm and term groups than sample, spectral, and fuzzy entropy. Entropy metrics provided complementary information to linear features, and indeed, the improvement in model performance by including other non-linear features was negligible. The best model performance obtained an F1-score of 90.1 ± 2% for testing the dataset. This model can easily be adapted to real-time applications, thereby contributing to the transferability of the EHG technique to clinical practice.

Phase trajectory entropy: A promising tool for passive diver detection

Passive diver detection is really significant as it constitutes a potential real-time monitoring of serious underwater threats. Up to now, there is still a lack of an efficient approach to characterize the complexity and fickleness with non-parametric and non-information priors. To achieve an improvement, a phase trajectory entropy method is proposed that should be promising. A coarser-grained distribution is created during entropy counting. The value of phase trajectory entropy is demonstrated by simulation and applied to real recorded data. The results show that phase trajectory entropy method considerably outperforms narrowband energy detection and the bubble entropy method.

A Two-Steps-Ahead Estimator for Bubble Entropy

Aims: Bubble entropy (bEn) is an entropy metric with a limited dependence on parameters. bEn does not directly quantify the conditional entropy of the series, but it assesses the change in entropy of the ordering of portions of its samples of length m, when adding an extra element. The analytical formulation of bEn for autoregressive (AR) processes shows that, for this class of processes, the relation between the first autocorrelation coefficient and bEn changes for odd and even values of m. While this is not an issue, per se, it triggered ideas for further investigation. Methods: Using theoretical considerations on the expected values for AR processes, we examined a two-steps-ahead estimator of bEn, which considered the cost of ordering two additional samples. We first compared it with the original bEn estimator on a simulated series. Then, we tested it on real heart rate variability (HRV) data. Results: The experiments showed that both examined alternatives showed comparable discriminating power. However, for values of 10 Collapse

Key Words

• bubble entropy
• entropy
• limited dependence on parameters

Collapse

MESH Headings Collapse

Grants Collapse

Affiliation(s)

George Manis Department of Computer Science and Engineering, University of Ioannina, 45500 Ioannina, Greece
Matteo Bodini Dipartimento di Informatica, Università degli Studi di Milano, 20133 Milan, Italy; (M.B.); (M.W.R.)
Massimo W. Rivolta Dipartimento di Informatica, Università degli Studi di Milano, 20133 Milan, Italy; (M.B.); (M.W.R.)
Roberto Sassi Dipartimento di Informatica, Università degli Studi di Milano, 20133 Milan, Italy; (M.B.); (M.W.R.)

Collapse

A Review of EEG Signal Features and their Application in Driver Drowsiness Detection Systems

Detecting drowsiness in drivers, especially multi-level drowsiness, is a difficult problem that is often approached using neurophysiological signals as the basis for building a reliable system. In this context, electroencephalogram (EEG) signals are the most important source of data to achieve successful detection. In this paper, we first review EEG signal features used in the literature for a variety of tasks, then we focus on reviewing the applications of EEG features and deep learning approaches in driver drowsiness detection, and finally we discuss the open challenges and opportunities in improving driver drowsiness detection based on EEG. We show that the number of studies on driver drowsiness detection systems has increased in recent years and that future systems need to consider the wide variety of EEG signal features and deep learning approaches to increase the accuracy of detection.

Automated Classification of Mental Arithmetic Tasks Using Recurrent Neural Network and Entropy Features Obtained from Multi-Channel EEG Signals

The automated classification of cognitive workload tasks based on the analysis of multi-channel EEG signals is vital for human–computer interface (HCI) applications. In this paper, we propose a computerized approach for categorizing mental-arithmetic-based cognitive workload tasks using multi-channel electroencephalogram (EEG) signals. The approach evaluates various entropy features, such as the approximation entropy, sample entropy, permutation entropy, dispersion entropy, and slope entropy, from each channel of the EEG signal. These features were fed to various recurrent neural network (RNN) models, such as long-short term memory (LSTM), bidirectional LSTM (BLSTM), and gated recurrent unit (GRU), for the automated classification of mental-arithmetic-based cognitive workload tasks. Two cognitive workload classification strategies (bad mental arithmetic calculation (BMAC) vs. good mental arithmetic calculation (GMAC); and before mental arithmetic calculation (BFMAC) vs. during mental arithmetic calculation (DMAC)) are considered in this work. The approach was evaluated using the publicly available mental arithmetic task-based EEG database. The results reveal that our proposed approach obtained classification accuracy values of 99.81%, 99.43%, and 99.81%, using the LSTM, BLSTM, and GRU-based RNN classifiers, respectively for the BMAC vs. GMAC cognitive workload classification strategy using all entropy features and a 10-fold cross-validation (CV) technique. The slope entropy features combined with each RNN-based model obtained higher classification accuracy compared with other entropy features for the classification of the BMAC vs. GMAC task. We obtained the average classification accuracy values of 99.39%, 99.44%, and 99.63% for the classification of the BFMAC vs. DMAC tasks, using the LSTM, BLSTM, and GRU classifiers with all entropy features and a hold-out CV scheme. Our developed automated mental arithmetic task system is ready to be tested with more databases for real-world applications.

CEPS: An Open Access MATLAB Graphical User Interface (GUI) for the Analysis of Complexity and Entropy in Physiological Signals

BACKGROUND

We developed CEPS as an open access MATLAB^® GUI (graphical user interface) for the analysis of Complexity and Entropy in Physiological Signals (CEPS), and demonstrate its use with an example data set that shows the effects of paced breathing (PB) on variability of heart, pulse and respiration rates. CEPS is also sufficiently adaptable to be used for other time series physiological data such as EEG (electroencephalography), postural sway or temperature measurements.

METHODS

Data were collected from a convenience sample of nine healthy adults in a pilot for a larger study investigating the effects on vagal tone of breathing paced at various different rates, part of a development programme for a home training stress reduction system.

RESULTS

The current version of CEPS focuses on those complexity and entropy measures that appear most frequently in the literature, together with some recently introduced entropy measures which may have advantages over those that are more established. Ten methods of estimating data complexity are currently included, and some 28 entropy measures. The GUI also includes a section for data pre-processing and standard ancillary methods to enable parameter estimation of embedding dimension m and time delay τ ('tau') where required. The software is freely available under version 3 of the GNU Lesser General Public License (LGPLv3) for non-commercial users. CEPS can be downloaded from Bitbucket. In our illustration on PB, most complexity and entropy measures decreased significantly in response to breathing at 7 breaths per minute, differentiating more clearly than conventional linear, time- and frequency-domain measures between breathing states. In contrast, Higuchi fractal dimension increased during paced breathing.

CONCLUSIONS

We have developed CEPS software as a physiological data visualiser able to integrate state of the art techniques. The interface is designed for clinical research and has a structure designed for integrating new tools. The aim is to strengthen collaboration between clinicians and the biomedical community, as demonstrated here by using CEPS to analyse various physiological responses to paced breathing.

The Entropy Universe

About 160 years ago, the concept of entropy was introduced in thermodynamics by Rudolf Clausius. Since then, it has been continually extended, interpreted, and applied by researchers in many scientific fields, such as general physics, information theory, chaos theory, data mining, and mathematical linguistics. This paper presents The Entropy Universe, which aims to review the many variants of entropies applied to time-series. The purpose is to answer research questions such as: How did each entropy emerge? What is the mathematical definition of each variant of entropy? How are entropies related to each other? What are the most applied scientific fields for each entropy? We describe in-depth the relationship between the most applied entropies in time-series for different scientific fields, establishing bases for researchers to properly choose the variant of entropy most suitable for their data. The number of citations over the past sixteen years of each paper proposing a new entropy was also accessed. The Shannon/differential, the Tsallis, the sample, the permutation, and the approximate entropies were the most cited ones. Based on the ten research areas with the most significant number of records obtained in the Web of Science and Scopus, the areas in which the entropies are more applied are computer science, physics, mathematics, and engineering. The universe of entropies is growing each day, either due to the introducing new variants either due to novel applications. Knowing each entropy's strengths and of limitations is essential to ensure the proper improvement of this research field.

Classification of Actigraphy Records from Bipolar Disorder Patients Using Slope Entropy: A Feasibility Study

Bipolar Disorder (BD) is an illness with high prevalence and a huge social and economic impact. It is recurrent, with a long-term evolution in most cases. Early treatment and continuous monitoring have proven to be very effective in mitigating the causes and consequences of BD. However, no tools are currently available for a massive and semi-automatic BD patient monitoring and control. Taking advantage of recent technological developments in the field of wearables, this paper studies the feasibility of a BD episodes classification analysis while using entropy measures, an approach successfully applied in a myriad of other physiological frameworks. This is a very difficult task, since actigraphy records are highly non-stationary and corrupted with artifacts (no activity). The method devised uses a preprocessing stage to extract epochs of activity, and then applies a quantification measure, Slope Entropy, recently proposed, which outperforms the most common entropy measures used in biomedical time series. The results confirm the feasibility of the approach proposed, since the three states that are involved in BD, depression, mania, and remission, can be significantly distinguished.

Development of Automated Sleep Stage Classification System Using Multivariate Projection-Based Fixed Boundary Empirical Wavelet Transform and Entropy Features Extracted from Multichannel EEG Signals

The categorization of sleep stages helps to diagnose different sleep-related ailments. In this paper, an entropy-based information–theoretic approach is introduced for the automated categorization of sleep stages using multi-channel electroencephalogram (EEG) signals. This approach comprises of three stages. First, the decomposition of multi-channel EEG signals into sub-band signals or modes is performed using a novel multivariate projection-based fixed boundary empirical wavelet transform (MPFBEWT) filter bank. Second, entropy features such as bubble and dispersion entropies are computed from the modes of multi-channel EEG signals. Third, a hybrid learning classifier based on class-specific residuals using sparse representation and distances from nearest neighbors is used to categorize sleep stages automatically using entropy-based features computed from MPFBEWT domain modes of multi-channel EEG signals. The proposed approach is evaluated using the multi-channel EEG signals obtained from the cyclic alternating pattern (CAP) sleep database. Our results reveal that the proposed sleep staging approach has obtained accuracies of 91.77%, 88.14%, 80.13%, and 73.88% for the automated categorization of wake vs. sleep, wake vs. rapid eye movement (REM) vs. Non-REM, wake vs. light sleep vs. deep sleep vs. REM sleep, and wake vs. S1-sleep vs. S2-sleep vs. S3-sleep vs. REM sleep schemes, respectively. The developed method has obtained the highest overall accuracy compared to the state-of-art approaches and is ready to be tested with more subjects before clinical application.

Using the Information Provided by Forbidden Ordinal Patterns in Permutation Entropy to Reinforce Time Series Discrimination Capabilities

Despite its widely tested and proven usefulness, there is still room for improvement in the basic permutation entropy (PE) algorithm, as several subsequent studies have demonstrated in recent years. Some of these new methods try to address the well-known PE weaknesses, such as its focus only on ordinal and not on amplitude information, and the possible detrimental impact of equal values found in subsequences. Other new methods address less specific weaknesses, such as the PE results' dependence on input parameter values, a common problem found in many entropy calculation methods. The lack of discriminating power among classes in some cases is also a generic problem when entropy measures are used for data series classification. This last problem is the one specifically addressed in the present study. Toward that purpose, the classification performance of the standard PE method was first assessed by conducting several time series classification tests over a varied and diverse set of data. Then, this performance was reassessed using a new Shannon Entropy normalisation scheme proposed in this paper: divide the relative frequencies in PE by the number of different ordinal patterns actually found in the time series, instead of by the theoretically expected number. According to the classification accuracy obtained, this last approach exhibited a higher class discriminating power. It was capable of finding significant differences in six out of seven experimental datasets-whereas the standard PE method only did in four-and it also had better classification accuracy. It can be concluded that using the additional information provided by the number of forbidden/found patterns, it is possible to achieve a higher discriminating power than using the classical PE normalisation method. The resulting algorithm is also very similar to that of PE and very easy to implement.

Slope Entropy: A New Time Series Complexity Estimator Based on Both Symbolic Patterns and Amplitude Information

The development of new measures and algorithms to quantify the entropy or related concepts of a data series is a continuous effort that has brought many innovations in this regard in recent years. The ultimate goal is usually to find new methods with a higher discriminating power, more efficient, more robust to noise and artifacts, less dependent on parameters or configurations, or any other possibly desirable feature. Among all these methods, Permutation Entropy (PE) is a complexity estimator for a time series that stands out due to its many strengths, with very few weaknesses. One of these weaknesses is the PE’s disregarding of time series amplitude information. Some PE algorithm modifications have been proposed in order to introduce such information into the calculations. We propose in this paper a new method, Slope Entropy (SlopEn), that also addresses this flaw but in a different way, keeping the symbolic representation of subsequences using a novel encoding method based on the slope generated by two consecutive data samples. By means of a thorough and extensive set of comparative experiments with PE and Sample Entropy (SampEn), we demonstrate that SlopEn is a very promising method with clearly a better time series classification performance than those previous methods.

Phase entropy: a new complexity measure for heart rate variability

OBJECTIVE

Information entropy is generally employed for analysing the complexity of physiological signals. However, most definitions of entropy estimate the degree of compressibility and thus quantify the randomness. Physiological signals are very complex because of nonlinear relationships and interactions between various systems and subsystems of the body. Therefore, analysis of randomness may not be sufficient to describe this complexity. To analyse the complexity of heart rate variability (HRV), a new entropy method, phase entropy (PhEn), has been proposed as a quantification of two-dimensional phase space.

APPROACH

The second-order difference plot (SODP), a two-dimensional phase space, provides a visual summary of the rate of variability. The distribution of scatter points in a SODP provides information about the dynamics of the underlying system. PhEn estimates the Shannon entropy of the weighted distribution in a coarse-grained SODP.

MAIN RESULTS

The performance of PhEn has been evaluated using simulated signals, synthetic HRV signals and real HRV signals. PhEn shows a better discriminating power and stability than other entropy measures. It is computationally efficient. Moreover, it has the ability to assess temporal asymmetry of physiological signals.

SIGNIFICANCE

PhEn quantifies the multiplicity and rate of variability associated with physiological signals. It is sensitive to time irreversibility. Therefore, it appears to be a promising tool for analysing physiological signals such as HRV.

Permutation Entropy: Enhancing Discriminating Power by Using Relative Frequencies Vector of Ordinal Patterns Instead of Their Shannon Entropy

Many measures to quantify the nonlinear dynamics of a time series are based on estimating the probability of certain features from their relative frequencies. Once a normalised histogram of events is computed, a single result is usually derived. This process can be broadly viewed as a nonlinear IRn mapping into IR, where n is the number of bins in the histogram. However, this mapping might entail a loss of information that could be critical for time series classification purposes. In this respect, the present study assessed such impact using permutation entropy (PE) and a diverse set of time series. We first devised a method of generating synthetic sequences of ordinal patterns using hidden Markov models. This way, it was possible to control the histogram distribution and quantify its influence on classification results. Next, real body temperature records are also used to illustrate the same phenomenon. The experiments results confirmed the improved classification accuracy achieved using raw histogram data instead of the PE final values. Thus, this study can provide a very valuable guidance for the improvement of the discriminating capability not only of PE, but of many similar histogram-based measures.

Obstructive Sleep Apnea Recognition Based on Multi-Bands Spectral Entropy Analysis of Short-Time Heart Rate Variability

Obstructive sleep apnea (OSA) syndrome is a common sleep disorder. As an alternative to polysomnography (PSG) for OSA screening, the current automatic OSA detection methods mainly concentrate on feature extraction and classifier selection based on physiological signals. It has been reported that OSA is, along with autonomic nervous system (ANS) dysfunction and heart rate variability (HRV), a useful tool for ANS assessment. Therefore, in this paper, eight novel indices of short-time HRV are extracted for OSA detection, which are based on the proposed multi-bands time-frequency spectrum entropy (MTFSE) method. In the MTFSE, firstly, the power spectrum of HRV is estimated by the Burg-AR model, and the time-frequency spectrum image (TFSI) is obtained. Secondly, according to the physiological significance of HRV, the TFSI is divided into multiple sub-bands according to frequency. Last but not least, by studying the Shannon entropy of different sub-bands and the relationships among them, the eight indices are obtained. In order to validate the performance of MTFSE-based indices, the Physionet Apnea-ECG database and K-nearest neighbor (KNN), support vector machine (SVM), and decision tree (DT) classification methods are used. The SVM classification method gets the highest classification accuracy, its average accuracy is 91.89%, the average sensitivity is 88.01%, and the average specificity is 93.98%. Undeniably, the MTFSE-based indices provide a novel idea for the screening of OSA disease.

Twenty Years of Entropy Research: A Bibliometric Overview

Entropy, founded in 1999, is an emerging international journal in the field of entropy and information studies. In the year of 2018, the journal enjoyed its 20th anniversary, and therefore, it is quite reasonable and meaningful to conduct a retrospective as its birthday gift. In accordance with Entropy’s distinctive name and research area, this paper creatively provides a bibliometric analysis method to not only look back at the vicissitude of the entire entropy topic, but also witness the journal’s growth and influence during this process. Based on 123,063 records extracted from the Web of Science, the work in sequence analyzes publication outputs, high-cited literature, and reference co-citation networks, in the aspects of the topic and the journal, respectively. The results indicate that the topic now has become a tremendous research domain and is still roaring ahead with great potentiality, widely researched by different kinds of disciplines. The most significant hotspots so far are suggested as the theoretical or practical innovation of graph entropy, permutation entropy, and pseudo-additive entropy. Furthermore, with the rapid growth in recent years, Entropy has attracted many dominant authors of the topic and experiences a distinctive geographical publication distribution. More importantly, in the midst of the topic, the journal has made enormous contributions to major research areas, particularly being a spear head in the studies of multiscale entropy and permutation entropy.

A hierarchical method based on weighted extreme gradient boosting in ECG heartbeat classification

BACKGROUND AND OBJECTIVE

Electrocardiogram (ECG) is a useful tool for detecting heart disease. Automated ECG diagnosis allows for heart monitoring on small devices, especially on wearable devices. In order to recognize arrhythmias automatically, accurate classification method for electrocardiogram (ECG) heartbeats was studied in this paper.

METHODS

Based on weighted extreme gradient boosting (XGBoost), a hierarchical classification method is proposed. A large number of features from 6 categories are extracted from the preprocessed heartbeats. Then recursive feature elimination is used for selecting features. Afterwards, a hierarchical classifier is constructed in classification stage. The hierarchical classifier is composed of threshold and XGBoost classifiers. And the XGBoost classifiers are improved with weights.

RESULTS

The method was applied to an inter-patient experiment conforming AAMI standard. The obtained sensitivities for normal (N), supraventricular (S), ventricular (V), fusion (F), and Unknown beats (Q) were 92.1%, 91.7%, 95.1%, and 61.6%. Positive predictive values of 99.5%, 46.2%, 88.1%, and 15.2% were also provided for the four classes.

CONCLUSIONS

XGBoost was improved and firstly introduced in single heartbeat classification. A comparison showed the effectiveness of the novel method. The method was more suitable for clinical application as both high positive predictive value for N class and high sensitivities for abnormal classes were provided.

Magnetotelluric Signal-Noise Identification and Separation Based on ApEn-MSE and StOMP

Natural magnetotelluric signals are extremely weak and susceptible to various types of noise pollution. To obtain more useful magnetotelluric data for further analysis and research, effective signal-noise identification and separation is critical. To this end, we propose a novel method of magnetotelluric signal-noise identification and separation based on ApEn-MSE and Stagewise orthogonal matching pursuit (StOMP). Parameters with good irregularity metrics are introduced: Approximate entropy (ApEn) and multiscale entropy (MSE), in combination with k-means clustering, can be used to accurately identify the data segments that are disturbed by noise. Stagewise orthogonal matching pursuit (StOMP) is used for noise suppression only in data segments identified as containing strong interference. Finally, we reconstructed the signal. The results show that the proposed method can better preserve the low-frequency slow-change information of the magnetotelluric signal compared with just using StOMP, thus avoiding the loss of useful information due to over-processing, while producing a smoother and more continuous apparent resistivity curve. Moreover, the results more accurately reflect the inherent electrical structure information of the measured site itself.

Classification of glucose records from patients at diabetes risk using a combined permutation entropy algorithm

BACKGROUND AND OBJECTIVES

The adoption in clinical practice of electronic portable blood or interstitial glucose monitors has enabled the collection, storage, and sharing of massive amounts of glucose level readings. This availability of data opened the door to the application of a multitude of mathematical methods to extract clinical information not discernible with conventional visual inspection. The objective of this study is to assess the capability of Permutation Entropy (PE) to find differences between glucose records of healthy and potentially diabetic subjects.

METHODS

PE is a mathematical method based on the relative frequency analysis of ordinal patterns in time series that has gained a lot of attention in the last years due to its simplicity, robustness, and performance. We study in this paper the applicability of this method to glucose records of subjects at risk of diabetes in order to assess the predictability value of this metric in this context.

RESULTS

PE, along with some of its derivatives, was able to find significant differences between diabetic and non-diabetic patients from records acquired up to 3 years before the diagnosis. The quantitative results for PE were 3.5878 ± 0.3916 for the nondiabetic class, and 3.1564 ± 0.4166 for the diabetic class. With a classification accuracy higher than 70%, and by means of a Cox regression model, PE demonstrated that it is a very promising candidate as a risk stratification tool for continuous glucose monitoring.

CONCLUSION

PE can be considered as a prospective tool for the early diagnosis of the glucoregulatory system.

Video Summarization for Sign Languages Using the Median of Entropy of Mean Frames Method

Multimedia information requires large repositories of audio-video data. Retrieval and delivery of video content is a very time-consuming process and is a great challenge for researchers. An efficient approach for faster browsing of large video collections and more efficient content indexing and access is video summarization. Compression of data through extraction of keyframes is a solution to these challenges. A keyframe is a representative frame of the salient features of the video. The output frames must represent the original video in temporal order. The proposed research presents a method of keyframe extraction using the mean of consecutive k frames of video data. A sliding window of size k/2 is employed to select the frame that matches the median entropy value of the sliding window. This is called the Median of Entropy of Mean Frames (MME) method. MME is mean-based keyframes selection using the median of the entropy of the sliding window. The method was tested for more than 500 videos of sign language gestures and showed satisfactory results.

Changes in the Complexity of Heart Rate Variability with Exercise Training Measured by Multiscale Entropy-Based Measurements

Quantifying complexity from heart rate variability (HRV) series is a challenging task, and multiscale entropy (MSE), along with its variants, has been demonstrated to be one of the most robust approaches to achieve this goal. Although physical training is known to be beneficial, there is little information about the long-term complexity changes induced by the physical conditioning. The present study aimed to quantify the changes in physiological complexity elicited by physical training through multiscale entropy-based complexity measurements. Rats were subject to a protocol of medium intensity training ( n = 13 ) or a sedentary protocol ( n = 12 ). One-hour HRV series were obtained from all conscious rats five days after the experimental protocol. We estimated MSE, multiscale dispersion entropy (MDE) and multiscale SDiff q from HRV series. Multiscale SDiff q is a recent approach that accounts for entropy differences between a given time series and its shuffled dynamics. From SDiff q , three attributes (q-attributes) were derived, namely SDiff q m a x , q m a x and q z e r o . MSE, MDE and multiscale q-attributes presented similar profiles, except for SDiff q m a x . q m a x showed significant differences between trained and sedentary groups on Time Scales 6 to 20. Results suggest that physical training increases the system complexity and that multiscale q-attributes provide valuable information about the physiological complexity.