Low Computational Cost for Sample Entropy

Sample Entropy Computation on Signals with Missing Values

Sample entropy embeds time series into m-dimensional spaces and estimates entropy based on the distances between points in these spaces. However, when samples can be considered as missing or invalid, defining distance in the embedding space becomes problematic. Preprocessing techniques, such as deletion or interpolation, can be employed as a solution, producing time series without missing or invalid values. While deletion ignores missing values, interpolation replaces them using approximations based on neighboring points. This paper proposes a novel approach for the computation of sample entropy when values are considered as missing or invalid. The proposed algorithm accommodates points in the m-dimensional space and handles them there. A theoretical and experimental comparison of the proposed algorithm with deletion and interpolation demonstrates several advantages over these other two approaches. Notably, the deviation of the expected sample entropy value for the proposed methodology consistently proves to be lowest one.

A Fast Algorithm for Estimating Two-Dimensional Sample Entropy Based on an Upper Confidence Bound and Monte Carlo Sampling

The two-dimensional sample entropy marks a significant advance in evaluating the regularity and predictability of images in the information domain. Unlike the direct computation of sample entropy, which incurs a time complexity of O(N2) for the series with N length, the Monte Carlo-based algorithm for computing one-dimensional sample entropy (MCSampEn) markedly reduces computational costs by minimizing the dependence on N. This paper extends MCSampEn to two dimensions, referred to as MCSampEn2D. This new approach substantially accelerates the estimation of two-dimensional sample entropy, outperforming the direct method by more than a thousand fold. Despite these advancements, MCSampEn2D encounters challenges with significant errors and slow convergence rates. To counter these issues, we have incorporated an upper confidence bound (UCB) strategy in MCSampEn2D. This strategy involves assigning varied upper confidence bounds in each Monte Carlo experiment iteration to enhance the algorithm's speed and accuracy. Our evaluation of this enhanced approach, dubbed UCBMCSampEn2D, involved the use of medical and natural image data sets. The experiments demonstrate that UCBMCSampEn2D achieves a 40% reduction in computational time compared to MCSampEn2D. Furthermore, the errors with UCBMCSampEn2D are only 30% of those observed in MCSampEn2D, highlighting its improved accuracy and efficiency.

A fast sample entropy for pulse rate variability analysis

Sample entropy is an effective nonlinear index for analyzing pulse rate variability (PRV) signal, but it has problems with a large amount of calculation and time consumption. Therefore, this study proposes a fast sample entropy calculation method to analyze the PRV signal according to the microprocessor process of data updating and the principle of sample entropy. The simulated data and PRV signal are employed as experimental data to verify the accuracy and time consumption of the proposed method. The experimental results on simulated data display that the proposed improved sample entropy can improve the operation rate of the entropy value by a maximum of 47.6 times and an average of 28.6 times and keep the entropy value unchanged. Experimental results on PRV signal display that the proposed improved sample entropy has great potential in the real-time processing of physiological signals, which can increase approximately 35 times.

Epileptic seizure prediction and classification based on statistical features using LSTM fully connected neural network

Epilepsy, the most common neurological disorder by which over 65 million people are affected across the world. Recent research has shown a very large interest to predict and diagnose epilepsy well before time. The continuous monitoring of EEG signals for seizure detection in electroencephalogram (EEG) is a very tedious and time taking process and therefore requires a qualified and trained clinical specialist. This paper presents a novel approach to detect and predict the epileptic signal in the recorded electroencephalogram (EEG). There is always a requirement for a nonlinear technique to examine the EEG signals due to the random nature of EEG signals. Therefore, we are providing an alternate method that extracts various entropy measures such Sample Entropy, Spectral Entropy, Permutation Entropy, and Shannon Entropy as statistical features from EEG signal. Based on these extracted features LSTM Fully connected Neural Network is used to classify the EEG signal as Focal and Non-focal. The proposed method gives a new insight into EEG signals by providing sensitivity as an added measure using deep learning along with accuracy and precision.

Evaluation of Fast Sample Entropy Algorithms on FPGAs: From Performance to Energy Efficiency

Entropy is one of the most fundamental notions for understanding complexity. Among all the methods to calculate the entropy, sample entropy (SampEn) is a practical and common method to estimate time-series complexity. Unfortunately, SampEn is a time-consuming method growing in quadratic times with the number of elements, which makes this method unviable when processing large data series. In this work, we evaluate hardware SampEn architectures to offload computation weight, using improved SampEn algorithms and exploiting reconfigurable technologies, such as field-programmable gate arrays (FPGAs), a reconfigurable technology well-known for its high performance and power efficiency. In addition to the fundamental disclosed straightforward SampEn (SF) calculation method, this study evaluates optimized strategies, such as bucket-assist (BA) SampEn and lightweight SampEn based on BubbleSort (BS-LW) and MergeSort (MS-LW) on an embedded CPU, a high-performance CPU and on an FPGA using simulated data and real-world electrocardiograms (ECG) as input data. Irregular storage space and memory access of enhanced algorithms is also studied and estimated in this work. These fast SampEn algorithms are evaluated and profiled using metrics such as execution time, resource use, power and energy consumption based on input data length. Finally, although the implementation of fast SampEn is not significantly faster than versions running on a high-performance CPU, FPGA implementations consume one or two orders of magnitude less energy than a high-performance CPU.

The Structure of Chaos: An Empirical Comparison of Fractal Physiology Complexity Indices Using NeuroKit2

Complexity quantification, through entropy, information theory and fractal dimension indices, is gaining a renewed traction in psychophsyiology, as new measures with promising qualities emerge from the computational and mathematical advances. Unfortunately, few studies compare the relationship and objective performance of the plethora of existing metrics, in turn hindering reproducibility, replicability, consistency, and clarity in the field. Using the NeuroKit2 Python software, we computed a list of 112 (predominantly used) complexity indices on signals varying in their characteristics (noise, length and frequency spectrum). We then systematically compared the indices by their computational weight, their representativeness of a multidimensional space of latent dimensions, and empirical proximity with other indices. Based on these considerations, we propose that a selection of 12 indices, together representing 85.97% of the total variance of all indices, might offer a parsimonious and complimentary choice in regards to the quantification of the complexity of time series. Our selection includes CWPEn, Line Length (LL), BubbEn, MSWPEn, MFDFA (Max), Hjorth Complexity, SVDEn, MFDFA (Width), MFDFA (Mean), MFDFA (Peak), MFDFA (Fluctuation), AttEn. Elements of consideration for alternative subsets are discussed, and data, analysis scripts and code for the figures are open-source.

Very Short-Term Photoplethysmography-Based Heart Rate Variability for Continuous Autoregulation Assessment

Background: Heart rate variability (HRV) has been widely applied for disease diagnosis. However, the 5 min signal length for HRV analysis is needed. Method: A signal processing procedure for very short-term photoplethysmography (PPG) signal for fever detection and autoregulation assessment was proposed. The Time-Shift Multiscale Entropy Analysis (TSME) was applied to instantaneous pulse rate time series (iPR) and normalized by the cumulative distribution function (CDF) of all scales to calculate novel indices. A total of 33 subjects were recruited for the study. Fifteen participants whose body temperatures were higher than 37.9 °C were served as the fever group. Others were served as the non-fever group. The total 15 s PPG signal with 200 sampling rates was used for iPR calculation. Result: The CDF value of entropy on the scale k = 19 (CDF(E(k = 19))) of iPR had the lowest p-value calculated by the Weltch t-test between two groups (p < 0.001). The Spearman correlation r between CDF(E(k = 19)) and body temperature is −0.757, 0.287, and −0.830 in all subjects, the non-fever group and the Fever group, respectively. The area under the curve, calculated from the receiver operating characteristic of CDF(E(k = 19)) of iPR is 0.915. Conclusion: The entropy of iPR is useful for detecting fever. Moreover, a short-term PPG signal is suitable to develop real-time applications, and multiscale entropy provides different scales of information for daily healthcare.

A Super Fast Algorithm for Estimating Sample Entropy

Sample entropy, an approximation of the Kolmogorov entropy, was proposed to characterize complexity of a time series, which is essentially defined as −log(B/A), where B denotes the number of matched template pairs with length m and A denotes the number of matched template pairs with m+1, for a predetermined positive integer m. It has been widely used to analyze physiological signals. As computing sample entropy is time consuming, the box-assisted, bucket-assisted, x-sort, assisted sliding box, and kd-tree-based algorithms were proposed to accelerate its computation. These algorithms require O(N2) or O(N2−1m+1) computational complexity, where N is the length of the time series analyzed. When N is big, the computational costs of these algorithms are large. We propose a super fast algorithm to estimate sample entropy based on Monte Carlo, with computational costs independent of N (the length of the time series) and the estimation converging to the exact sample entropy as the number of repeating experiments becomes large. The convergence rate of the algorithm is also established. Numerical experiments are performed for electrocardiogram time series, electroencephalogram time series, cardiac inter-beat time series, mechanical vibration signals (MVS), meteorological data (MD), and 1/f noise. Numerical results show that the proposed algorithm can gain 100–1000 times speedup compared to the kd-tree and assisted sliding box algorithms while providing satisfactory approximate accuracy.

Multi-Level Optimization of an Ultra-Low Power BrainWave System for Non-Convulsive Seizure Detection

We present a systematic evaluation and optimization of a complex bio-medical signal processing application on the BrainWave prototype system, targeted towards ambulatory EEG monitoring within a tiny power budget of 1 mW. The considered BrainWave processor is completely programmable, while maintaining energy-efficiency by means of a Coarse-Grained Reconfigurable Array (CGRA). This is demonstrated through the mapping and evaluation of a state-of-the-art non-convulsive epileptic seizure detection algorithm, while ensuring real-time operation and seizure detection accuracy. Exploiting the CGRA leads to an energy reduction of 73.1%, compared to a highly tuned software implementation (SW-only). A total of 9 complex kernels were benchmarked on the CGRA, resulting in an average 4.7 × speedup and average 4.4 × energy savings over highly tuned SW-only implementations. The BrainWave processor is implemented in 28-nm FDSOI technology with 80 kB of Foundry-provided SRAM. By exploiting near-threshold computing for the logic and voltage-stacking to minimize on-chip voltage-conversion overhead, additional 15.2% and 19.5% energy savings are obtained, respectively. At the Minimum-Energy-Point (MEP) (223 μW, 8 MHz) we report a measured state-of-the-art 90.6% system conversion efficiency, while executing the epileptic seizure detection in real-time.

Magnetotelluric Signal-Noise Identification and Separation Based on ApEn-MSE and StOMP

Natural magnetotelluric signals are extremely weak and susceptible to various types of noise pollution. To obtain more useful magnetotelluric data for further analysis and research, effective signal-noise identification and separation is critical. To this end, we propose a novel method of magnetotelluric signal-noise identification and separation based on ApEn-MSE and Stagewise orthogonal matching pursuit (StOMP). Parameters with good irregularity metrics are introduced: Approximate entropy (ApEn) and multiscale entropy (MSE), in combination with k-means clustering, can be used to accurately identify the data segments that are disturbed by noise. Stagewise orthogonal matching pursuit (StOMP) is used for noise suppression only in data segments identified as containing strong interference. Finally, we reconstructed the signal. The results show that the proposed method can better preserve the low-frequency slow-change information of the magnetotelluric signal compared with just using StOMP, thus avoiding the loss of useful information due to over-processing, while producing a smoother and more continuous apparent resistivity curve. Moreover, the results more accurately reflect the inherent electrical structure information of the measured site itself.

Partial Discharge Fault Diagnosis Based on Multi-Scale Dispersion Entropy and a Hypersphere Multiclass Support Vector Machine

Partial discharge (PD) fault analysis is an important tool for insulation condition diagnosis of electrical equipment. In order to conquer the limitations of traditional PD fault diagnosis, a novel feature extraction approach based on variational mode decomposition (VMD) and multi-scale dispersion entropy (MDE) is proposed. Besides, a hypersphere multiclass support vector machine (HMSVM) is used for PD pattern recognition with extracted PD features. Firstly, the original PD signal is decomposed with VMD to obtain intrinsic mode functions (IMFs). Secondly proper IMFs are selected according to central frequency observation and MDE values in each IMF are calculated. And then principal component analysis (PCA) is introduced to extract effective principle components in MDE. Finally, the extracted principle factors are used as PD features and sent to HMSVM classifier. Experiment results demonstrate that, PD feature extraction method based on VMD-MDE can extract effective characteristic parameters that representing dominant PD features. Recognition results verify the effectiveness and superiority of the proposed PD fault diagnosis method.

Dynamic Complexity of Spontaneous BOLD Activity in Alzheimer's Disease and Mild Cognitive Impairment Using Multiscale Entropy Analysis

Alzheimer's disease (AD) is characterized by progressive deterioration of brain function among elderly people. Studies revealed aberrant correlations in spontaneous blood oxygen level-dependent (BOLD) signals in resting-state functional magnetic resonance imaging (rs-fMRI) over a wide range of temporal scales. However, the study of the temporal dynamics of BOLD signals in subjects with AD and mild cognitive impairment (MCI) remains largely unexplored. Multiscale entropy (MSE) analysis is a method for estimating the complexity of finite time series over multiple time scales. In this research, we applied MSE analysis to investigate the abnormal complexity of BOLD signals using the rs-fMRI data from the Alzheimer's disease neuroimaging initiative (ADNI) database. There were 30 normal controls (NCs), 33 early MCI (EMCI), 32 late MCI (LMCI), and 29 AD patients. Following preprocessing of the BOLD signals, whole-brain MSE maps across six time scales were generated using the Complexity Toolbox. One-way analysis of variance (ANOVA) analysis on the MSE maps of four groups revealed significant differences in the thalamus, insula, lingual gyrus and inferior occipital gyrus, superior frontal gyrus and olfactory cortex, supramarginal gyrus, superior temporal gyrus, and middle temporal gyrus on multiple time scales. Compared with the NC group, MCI and AD patients had significant reductions in the complexity of BOLD signals and AD patients demonstrated lower complexity than that of the MCI subjects. Additionally, the complexity of BOLD signals from the regions of interest (ROIs) was found to be significantly associated with cognitive decline in patient groups on multiple time scales. Consequently, the complexity or MSE of BOLD signals may provide an imaging biomarker of cognitive impairments in MCI and AD.

Sparse Optimistic Based on Lasso-LSQR and Minimum Entropy De-Convolution with FARIMA for the Remaining Useful Life Prediction of Machinery

To reduce the maintenance cost and safeguard machinery operation, remaining useful life (RUL) prediction is very important for long term health monitoring. In this paper, we introduce a novel hybrid method to deal with the RUL prediction for health management. Firstly, the sparse reconstruction algorithm of the optimized Lasso and the Least Square QR-factorization (Lasso-LSQR) is applied to compressed sensing (CS), which can realize the sparse optimization for long term health monitoring data. After the sparse signal is reconstructed, the minimum entropy de-convolution (MED) is used to identify the fault characteristics and to obtain significant fault information from the machinery operation. Health indicators with Skip-over, sample entropy and approximate entropy are then performed to track the degradation of the machinery process. The performance analysis of the Skip-over is superior to other indicators. Finally, Fractal Autoregressive Integrated Moving Average model (FARIMA) is employed to predict the Skip-over using the R/S method. The analysis results evidence that the novel hybrid method yields a good performance, and such method can achieve highly accurate RUL prediction and safeguard machinery operation for long term monitoring.

Fuzzy and Sample Entropies as Predictors of Patient Survival Using Short Ventricular Fibrillation Recordings during out of Hospital Cardiac Arrest

Optimal defibrillation timing guided by ventricular fibrillation (VF) waveform analysis would contribute to improved survival of out-of-hospital cardiac arrest (OHCA) patients by minimizing myocardial damage caused by futile defibrillation shocks and minimizing interruptions to cardiopulmonary resuscitation. Recently, fuzzy entropy (FuzzyEn) tailored to jointly measure VF amplitude and regularity has been shown to be an efficient defibrillation success predictor. In this study, 734 shocks from 296 OHCA patients (50 survivors) were analyzed, and the embedding dimension (m) and matching tolerance (r) for FuzzyEn and sample entropy (SampEn) were adjusted to predict defibrillation success and patient survival. Entropies were significantly larger in successful shocks and in survivors, and when compared to the available methods, FuzzyEn presented the best prediction results, marginally outperforming SampEn. The sensitivity and specificity of FuzzyEn were 83.3% and 76.7% when predicting defibrillation success, and 83.7% and 73.5% for patient survival. Sensitivities and specificities were two points above those of the best available methods, and the prediction accuracy was kept even for VF intervals as short as 2s. These results suggest that FuzzyEn and SampEn may be promising tools for optimizing the defibrillation time and predicting patient survival in OHCA patients presenting VF.

Hierarchical Cosine Similarity Entropy for Feature Extraction of Ship-Radiated Noise

The classification performance of passive sonar can be improved by extracting the features of ship-radiated noise. Traditional feature extraction methods neglect the nonlinear features in ship-radiated noise, such as entropy. The multiscale sample entropy (MSE) algorithm has been widely used for quantifying the entropy of a signal, but there are still some limitations. To remedy this, the hierarchical cosine similarity entropy (HCSE) is proposed in this paper. Firstly, the hierarchical decomposition is utilized to decompose a time series into some subsequences. Then, the sample entropy (SE) is modified by utilizing Shannon entropy rather than conditional entropy and employing angular distance instead of Chebyshev distance. Finally, the complexity of each subsequence is quantified by the modified SE. Simulation results show that the HCSE method overcomes some limitations in MSE. For example, undefined entropy is not likely to occur in HCSE, and it is more suitable for short time series. Compared with MSE, the experimental results illustrate that the classification accuracy of real ship-radiated noise is significantly improved from 75% to 95.63% by using HCSE. Consequently, the proposed HCSE can be applied in practical applications.

Characterization of the Stroke-Induced Changes in the Variability and Complexity of Handgrip Force

Introduction: The variability and complexity of handgrip forces in various modulations were investigated to identify post-stroke changes in force modulation, and extend our understanding of stroke-induced deficits. Methods: Eleven post-stroke subjects and ten age-matched controls performed voluntary grip force control tasks (power-grip tasks) at three contraction levels, and stationary dynamometer holding tasks (stationary holding tasks). Variability and complexity were described with root mean square jerk (RMS-jerk) and fuzzy approximate entropy (fApEn), respectively. Force magnitude, Fugl-Meyer upper extremity assessment and Wolf motor function test were also evaluated. Results: Comparing the affected side with the controls, fApEn was significantly decreased and RMS-jerk increased across the three levels in power-grip tasks, and fApEn was significantly decreased in stationary holding tasks. There were significant strong correlations between RMS-jerk and clinical scales in power-grip tasks. Discussion: Abnormal neuromuscular control, altered mechanical properties, and atrophic motoneurons could be the main causes of the differences in complexity and variability in post-stroke subjects.

Modulation Signal Recognition Based on Information Entropy and Ensemble Learning

In this paper, information entropy and ensemble learning based signal recognition theory and algorithms have been proposed. We have extracted 16 kinds of entropy features out of 9 types of modulated signals. The types of information entropy used are numerous, including Rényi entropy and energy entropy based on S Transform and Generalized S Transform. We have used three feature selection algorithms, including sequence forward selection (SFS), sequence forward floating selection (SFFS) and RELIEF-F to select the optimal feature subset from 16 entropy features. We use five classifiers, including k-nearest neighbor (KNN), support vector machine (SVM), Adaboost, Gradient Boosting Decision Tree (GBDT) and eXtreme Gradient Boosting (XGBoost) to classify the original feature set and the feature subsets selected by different feature selection algorithms. The simulation results show that the feature subsets selected by SFS and SFFS algorithms are the best, with a 48% increase in recognition rate over the original feature set when using KNN classifier and a 34% increase when using SVM classifier. For the other three classifiers, the original feature set can achieve the best recognition performance. The XGBoost classifier has the best recognition performance, the overall recognition rate is 97.74% and the recognition rate can reach 82% when the signal to noise ratio (SNR) is −10 dB.