Extracting strong measurement noise from stochastic time series: applications to empirical data

Reconstruction of stochastic dynamics from large streamed datasets

The complex dynamics of physical systems can often be modeled with stochastic differential equations. However, computational constraints inhibit the estimation of dynamics from large time-series datasets. I present a method for estimating drift and diffusion functions from inordinately large datasets through the use of incremental, online, updating statistics. I demonstrate the validity and utility of this method by analyzing three large, varied synthetic datasets, as well as an empirical turbulence dataset. This method will hopefully facilitate the analysis of complex systems from exceedingly large, "big data" scientific datasets, as well as real-time streamed data.

Estimation of drift and diffusion functions from unevenly sampled time-series data

Complex systems can often be modeled as stochastic processes. However, physical observations of such systems are often irregularly spaced in time, leading to difficulties in estimating appropriate models from data. Here we present extensions of two methods for estimating drift and diffusion functions from irregularly sampled time-series data. Our methods are flexible and applicable to a variety of stochastic systems, including non-Markov processes or systems contaminated with measurement noise. To demonstrate applicability, we use this approach to analyze an irregularly sampled paleoclimatological isotope record, giving insights into underlying physical processes.

Exit time as a measure of ecological resilience

Ecological resilience is the magnitude of the largest perturbation from which a system can still recover to its original state. However, a transition into another state may often be invoked by a series of minor synergistic perturbations rather than a single big one. We show how resilience can be estimated in terms of average life expectancy, accounting for this natural regime of variability. We use time series to fit a model that captures the stochastic as well as the deterministic components. The model is then used to estimate the mean exit time from the basin of attraction. This approach offers a fresh angle to anticipating the chance of a critical transition at a time when high-resolution time series are becoming increasingly available.

Analysis and data-driven reconstruction of bivariate jump-diffusion processes

We introduce the bivariate jump-diffusion process, consisting of two-dimensional diffusion and two-dimensional jumps, that can be coupled to one another. We present a data-driven, nonparametric estimation procedure of higher-order (up to 8) Kramers-Moyal coefficients that allows one to reconstruct relevant aspects of the underlying jump-diffusion processes and to recover the underlying parameters. The procedure is validated with numerically integrated data using synthetic bivariate time series from continuous and discontinuous processes. We further evaluate the possibility of estimating the parameters of the jump-diffusion model via data-driven analyses of the higher-order Kramers-Moyal coefficients, and the limitations arising from the scarcity of points in the data or disproportionate parameters in the system.

Robust identification of harmonic oscillator parameters using the adjoint Fokker-Planck equation

We present a model-based output-only method for identifying from time series the parameters governing the dynamics of stochastically forced oscillators. In this context, suitable models of the oscillator's damping and stiffness properties are postulated, guided by physical understanding of the oscillatory phenomena. The temporal dynamics and the probability density function of the oscillation amplitude are described by a Langevin equation and its associated Fokker-Planck equation, respectively. One method consists in fitting the postulated analytical drift and diffusion coefficients with their estimated values, obtained from data processing by taking the short-time limit of the first two transition moments. However, this limit estimation loses robustness in some situations-for instance when the data are band-pass filtered to isolate the spectral contents of the oscillatory phenomena of interest. In this paper, we use a robust alternative where the adjoint Fokker-Planck equation is solved to compute Kramers-Moyal coefficients exactly, and an iterative optimization yields the parameters that best fit the observed statistics simultaneously in a wide range of amplitudes and time scales. The method is illustrated with a stochastic Van der Pol oscillator serving as a prototypical model of thermoacoustic instabilities in practical combustors, where system identification is highly relevant to control.

Simple noise-reduction method based on nonlinear forecasting

Nonparametric detrending or noise reduction methods are often employed to separate trends from noisy time series when no satisfactory models exist to fit the data. However, conventional noise reduction methods depend on subjective choices of smoothing parameters. Here we present a simple multivariate noise reduction method based on available nonlinear forecasting techniques. These are in turn based on state-space reconstruction for which a strong theoretical justification exists for their use in nonparametric forecasting. The noise reduction method presented here is conceptually similar to Schreiber's noise reduction method using state-space reconstruction. However, we show that Schreiber's method has a minor flaw that can be overcome with forecasting. Furthermore, our method contains a simple but nontrivial extension to multivariate time series. We apply the method to multivariate time series generated from the Van der Pol oscillator, the Lorenz equations, the Hindmarsh-Rose model of neuronal spiking activity, and to two other univariate real-world data sets. It is demonstrated that noise reduction heuristics can be objectively optimized with in-sample forecasting errors that correlate well with actual noise reduction errors.

Analyzing a stochastic time series obeying a second-order differential equation

The stochastic properties of a Langevin-type Markov process can be extracted from a given time series by a Markov analysis. Also processes that obey a stochastically forced second-order differential equation can be analyzed this way by employing a particular embedding approach: To obtain a Markovian process in 2N dimensions from a non-Markovian signal in N dimensions, the system is described in a phase space that is extended by the temporal derivative of the signal. For a discrete time series, however, this derivative can only be calculated by a differencing scheme, which introduces an error. If the effects of this error are not accounted for, this leads to systematic errors in the estimation of the drift and diffusion functions of the process. In this paper we will analyze these errors and we will propose an approach that correctly accounts for them. This approach allows an accurate parameter estimation and, additionally, is able to cope with weak measurement noise, which may be superimposed to a given time series.

Uncovering wind turbine properties through two-dimensional stochastic modeling of wind dynamics

Using a method for stochastic data analysis borrowed from statistical physics, we analyze synthetic data from a Markov chain model that reproduces measurements of wind speed and power production in a wind park in Portugal. We show that our analysis retrieves indeed the power performance curve, which yields the relationship between wind speed and power production, and we discuss how this procedure can be extended for extracting unknown functional relationships between pairs of physical variables in general. We also show how specific features, such as the rated speed of the wind turbine or the descriptive wind speed statistics, can be related to the equations describing the evolution of power production and wind speed at single wind turbines.

Estimation of drift and diffusion functions from time series data: a maximum likelihood framework

Complex systems are characterized by a huge number of degrees of freedom often interacting in a nonlinear manner. In many cases macroscopic states, however, can be characterized by a small number of order parameters that obey stochastic dynamics in time. Recently, techniques for the estimation of the corresponding stochastic differential equations from measured data have been introduced. This paper develops a framework for the estimation of the functions and their respective (Bayesian posterior) confidence regions based on likelihood estimators. In succession, approximations are introduced that significantly improve the efficiency of the estimation procedure. While being consistent with standard approaches to the problem, this paper solves important problems concerning the applicability and the accuracy of estimated parameters.

Principal axes for stochastic dynamics

We introduce a general procedure for directly ascertaining how many independent stochastic sources exist in a complex system modeled through a set of coupled Langevin equations of arbitrary dimension. The procedure is based on the computation of the eigenvalues and the corresponding eigenvectors of local diffusion matrices. We demonstrate our algorithm by applying it to two examples of systems showing Hopf bifurcation. We argue that computing the eigenvectors associated to the eigenvalues of the diffusion matrix at local mesh points in the phase space enables one to define vector fields of stochastic eigendirections. In particular, the eigenvector associated to the lowest eigenvalue defines the path of minimum stochastic forcing in phase space, and a transform to a new coordinate system aligned with the eigenvectors can increase the predictability of the system.

Analysis of stochastic time series in the presence of strong measurement noise

An alternative approach for the analysis of Langevin-type stochastic processes in the presence of strong measurement noise is presented. For the case of Gaussian distributed, exponentially correlated, measurement noise it is possible to extract the strength and the correlation time of the noise as well as polynomial approximations of the drift and diffusion functions of the underlying Langevin equation.