Statistically relaxing to generating partitions for observed time-series data

A review of symbolic dynamics and symbolic reconstruction of dynamical systems

Discretizing a nonlinear time series enables us to calculate its statistics fast and rigorously. Before the turn of the century, the approach using partitions was dominant. In the last two decades, discretization via permutations has been developed to a powerful methodology, while recurrence plots have recently begun to be recognized as a method of discretization. In the meantime, horizontal visibility graphs have also been proposed to discretize time series. In this review, we summarize these methods and compare them from the viewpoint of symbolic dynamics, which is the right framework to study the symbolic representation of nonlinear time series and the inverse process: the symbolic reconstruction of dynamical systems. As we will show, symbolic dynamics is currently a very active research field with interesting applications.

A Spherical Phase Space Partitioning Based Symbolic Time Series Analysis (SPSP—STSA) for Emotion Recognition Using EEG Signals

Emotion recognition systems have been of interest to researchers for a long time. Improvement of brain-computer interface systems currently makes EEG-based emotion recognition more attractive. These systems try to develop strategies that are capable of recognizing emotions automatically. There are many approaches due to different features extractions methods for analyzing the EEG signals. Still, Since the brain is supposed to be a nonlinear dynamic system, it seems a nonlinear dynamic analysis tool may yield more convenient results. A novel approach in Symbolic Time Series Analysis (STSA) for signal phase space partitioning and symbol sequence generating is introduced in this study. Symbolic sequences have been produced by means of spherical partitioning of phase space; then, they have been compared and classified based on the maximum value of a similarity index. Obtaining the automatic independent emotion recognition EEG-based system has always been discussed because of the subject-dependent content of emotion. Here we introduce a subject-independent protocol to solve the generalization problem. To prove our method’s effectiveness, we used the DEAP dataset, and we reached an accuracy of 98.44% for classifying happiness from sadness (two- emotion groups). It was 93.75% for three (happiness, sadness, and joy), 89.06% for four (happiness, sadness, joy, and terrible), and 85% for five emotional groups (happiness, sadness, joy, terrible and mellow). According to these results, it is evident that our subject-independent method is more accurate rather than many other methods in different studies. In addition, a subject-independent method has been proposed in this study, which is not considered in most of the studies in this field.

Symbolic partition in chaotic maps

In this work, we only use data on the unstable manifold to locate the partition boundaries by checking folding points at different levels, which practically coincide with homoclinic tangencies. The method is then applied to the classic two-dimensional Hénon map and a well-known three-dimensional map. Comparison with previous results is made in the Hénon case, and Lyapunov exponents are computed through the metric entropy based on the partition to show the validity of the current scheme.

A Locally Optimal Algorithm for Estimating a Generating Partition from an Observed Time Series and Its Application to Anomaly Detection

Estimation of a generating partition is critical for symbolization of measurements from discrete-time dynamical systems, where a sequence of symbols from a (finite-cardinality) alphabet may uniquely specify the underlying time series. Such symbolization is useful for computing measures (e.g., Kolmogorov-Sinai entropy) to identify or characterize the (possibly unknown) dynamical system. It is also useful for time series classification and anomaly detection. The seminal work of Hirata, Judd, and Kilminster ( 2004 ) derives a novel objective function, akin to a clustering objective, that measures the discrepancy between a set of reconstruction values and the points from the time series. They cast estimation of a generating partition via the minimization of their objective function. Unfortunately, their proposed algorithm is nonconvergent, with no guarantee of finding even locally optimal solutions with respect to their objective. The difficulty is a heuristic nearest neighbor symbol assignment step. Alternatively, we develop a novel, locally optimal algorithm for their objective. We apply iterative nearest-neighbor symbol assignments with guaranteed discrepancy descent, by which joint, locally optimal symbolization of the entire time series is achieved. While most previous approaches frame generating partition estimation as a state-space partitioning problem, we recognize that minimizing the Hirata et al. ( 2004 ) objective function does not induce an explicit partitioning of the state space, but rather the space consisting of the entire time series (effectively, clustering in a (countably) infinite-dimensional space). Our approach also amounts to a novel type of sliding block lossy source coding. Improvement, with respect to several measures, is demonstrated over popular methods for symbolizing chaotic maps. We also apply our approach to time-series anomaly detection, considering both chaotic maps and failure application in a polycrystalline alloy material.

Quantifying the Dynamical Complexity of Chaotic Time Series

A powerful approach is proposed for the characterization of chaotic signals. It is based on the combined use of two classes of indicators: (i) the probability of suitable symbolic sequences (obtained from the ordinal patterns of the corresponding time series); (ii) the width of the corresponding cylinder sets. This way, much information can be extracted and used to quantify the complexity of a given signal. As an example of the potentiality of the method, I introduce a modified permutation entropy which allows for quantitative estimates of the Kolmogorov-Sinai entropy in hyperchaotic models, where other methods would be unpractical. As a by-product, estimates of the fractal dimension of the underlying attractors are possible as well.

Neighborhoods of periodic orbits and the stationary distribution of a noisy chaotic system

The finest state-space resolution that can be achieved in a physical dynamical system is limited by the presence of noise. In the weak-noise approximation, the stochastic neighborhoods of deterministic periodic orbits can be computed from distributions stationary under the action of a local Fokker-Planck operator and its adjoint. We derive explicit formulas for widths of these distributions in the case of chaotic dynamics, when the periodic orbits are hyperbolic. The resulting neighborhoods form a basis for functions on the attractor. The global stationary distribution, needed for calculation of long-time expectation values of observables, can be expressed in this basis.

Symbolic dynamic filtering and language measure for behavior identification of mobile robots

This paper presents a procedure for behavior identification of mobile robots, which requires limited or no domain knowledge of the underlying process. While the features of robot behavior are extracted by symbolic dynamic filtering of the observed time series, the behavior patterns are classified based on language measure theory. The behavior identification procedure has been experimentally validated on a networked robotic test bed by comparison with commonly used tools, namely, principal component analysis for feature extraction and Bayesian risk analysis for pattern classification.

Unstable periodic orbits and noise in chaos computing

Different methods to utilize the rich library of patterns and behaviors of a chaotic system have been proposed for doing computation or communication. Since a chaotic system is intrinsically unstable and its nearby orbits diverge exponentially from each other, special attention needs to be paid to the robustness against noise of chaos-based approaches to computation. In this paper unstable periodic orbits, which form the skeleton of any chaotic system, are employed to build a model for the chaotic system to measure the sensitivity of each orbit to noise, and to select the orbits whose symbolic representations are relatively robust against the existence of noise. Furthermore, since unstable periodic orbits are extractable from time series, periodic orbit-based models can be extracted from time series too. Chaos computing can be and has been implemented on different platforms, including biological systems. In biology noise is always present; as a result having a clear model for the effects of noise on any given biological implementation has profound importance. Also, since in biology it is hard to obtain exact dynamical equations of the system under study, the time series techniques we introduce here are of critical importance.

Computational mechanics of molecular systems: Quantifying high-dimensional dynamics by distribution of Poincaré recurrence times

A framework that connects computational mechanics and molecular dynamics has been developed and described. As the key parts of the framework, the problem of symbolising molecular trajectory and the associated interrelation between microscopic phase space variables and macroscopic observables of the molecular system are considered. Following Shalizi and Moore, it is shown that causal states, the constituent parts of the main construct of computational mechanics, the ε-machine, define areas of the phase space that are optimal in the sense of transferring information from the micro-variables to the macro-observables. We have demonstrated that, based on the decay of their Poincaré return times, these areas can be divided into two classes that characterise the separation of the phase space into resonant and chaotic areas. The first class is characterised by predominantly short time returns, typical to quasi-periodic or periodic trajectories. This class includes a countable number of areas corresponding to resonances. The second class includes trajectories with chaotic behaviour characterised by the exponential decay of return times in accordance with the Poincaré theorem.

Complex temporal patterns in molecular dynamics: a direct measure of the phase-space exploration by the trajectory at macroscopic time scales

Computer simulated trajectories of bulk water molecules form complex spatiotemporal structures at the picosecond time scale. This intrinsic complexity, which underlies the formation of molecular structures at longer time scales, has been quantified using a measure of statistical complexity. The method estimates the information contained in the molecular trajectory by detecting and quantifying temporal patterns present in the simulated data (velocity time series). Two types of temporal patterns are found. The first, defined by the short-time correlations corresponding to the velocity autocorrelation decay times (< or = 0.1 ps), remains asymptotically stable for time intervals longer than several tens of nanoseconds. The second is caused by previously unknown longer-time correlations (found at longer than the nanoseconds time scales) leading to a value of statistical complexity that slowly increases with time. A direct measure based on the notion of statistical complexity that describes how the trajectory explores the phase space and independent from the particular molecular signal used as the observed time series is introduced.

Optimal instruments and models for noisy chaos

Analysis of finite, noisy time series data leads to modern statistical inference methods. Here we adapt Bayesian inference for applied symbolic dynamics. We show that reconciling Kolmogorov's maximum-entropy partition with the methods of Bayesian model selection requires the use of two separate optimizations. First, instrument design produces a maximum-entropy symbolic representation of time series data. Second, Bayesian model comparison with a uniform prior selects a minimum-entropy model, with respect to the considered Markov chain orders, of the symbolic data. We illustrate these steps using a binary partition of time series data from the logistic and Henon maps as well as the Rössler and Lorenz attractors with dynamical noise. In each case we demonstrate the inference of effectively generating partitions and kth-order Markov chain models.

Globally enumerating unstable periodic orbits for observed data using symbolic dynamics

The unstable periodic orbits of a chaotic system provide an important skeleton of the dynamics in a chaotic system, but they can be difficult to find from an observed time series. We present a global method for finding periodic orbits based on their symbolic dynamics, which is made possible by several recent methods to find good partitions for symbolic dynamics from observed time series. The symbolic dynamics are approximated by a Markov chain estimated from the sequence using information-theoretical concepts. The chain has a probabilistic graph representation, and the cycles of the graph may be exhaustively enumerated with a classical deterministic algorithm, providing a global, comprehensive list of symbolic names for its periodic orbits. Once the symbolic codes of the periodic orbits are found, the partition is used to localize the orbits back in the original state space. Using the periodic orbits found, we can estimate several quantities of the attractor such as the Lyapunov exponent and topological entropy.

On parameter estimation of chaotic systems via symbolic time-series analysis

Symbolic time-series analysis is used for estimating the parameters of chaotic systems. It is assumed that a "target model" (i.e., a discrete- or continuous-time description of the data-generating mechanism) is available, but with unknown parameters. A time series, i.e., a noisy, finite sequence of a measured (output) variable, is given. The proposed method first prescribes to symbolize the time series, i.e., to transform it into a sequence of symbols, from which the statistics of symbols are readily derived. Then, a symbolic model (in the form of a Markov chain) is derived from the data. It allows one to predict, in a probabilistic fashion, the time evolution of the symbol sequence. The unknown parameters are derived by matching either the statistics of symbols, or the symbolic prediction derived from data, with those generated by the (parametrized) target model. Three examples of application (the Henon map, a population model, and the Duffing system) prove that satisfactory results can be obtained even with short time series.