Data-driven prediction in dynamical systems: recent developments

Data-driven approach to weekly forecast of the western flower thrips (Frankliniella occidentalis Pergande) population in a pepper greenhouse with an ensemble model

BACKGROUND

Integrated pest management (IPM) in European glasshouses has substantially advanced in automated insect pest detection systems lately. However, transforming such an enormous data influx into optimal biological control strategies remains challenging. In addition, most biological control forecast studies relied on the single-best model approach, which is susceptible to overconfidence, and they lack validation over sufficient sampling repetitions where robustness remains questionable. Here we propose employing an unweighted ensemble model, by combining multiple forecasting models ranging from simple models (linear regressions and Lotka-Volterra model) to machine learning models (Gaussian process, Random Forest, XGBoost, Multi-Layer Perceptron), to predict 1-week-ahead population of western flower thrips (Frankliniella occidentalis), a notorious pest in glasshouses, under the influence of its biological control agent Macrolophus pygmaeus in pepper-growing glasshouses.

RESULTS

Models were trained with only 1 year of data, validated over 3 years of monitoring of multiple compartments to evaluate their robustness. The full ensemble model outperformed the Naïve Forecast in 10 out of 14 compartments for validation, with around 0.451 and 26.6% increase in coefficient of determination (R2) and directional accuracy, respectively. It also extended 0.096 in R2 from the best single model, equivalent to a 27% increase in accuracy, while maintaining a 75% directional accuracy.

CONCLUSION

Our results demonstrated the benefits of the ensemble model over the traditional 'single-best model' approach, avoiding model structural biases and minimizing the risk of overconfidence. This showcased how an ensemble model with minimal training data can assist growers in fully utilizing the pest monitoring data and support their decision-making on IPM. © 2025 Society of Chemical Industry.

Data-driven nonlinear model reduction to spectral submanifolds via oblique projection

The dynamics in a primary spectral submanifold (SSM) constructed over the slowest modes of a dynamical system provide an ideal reduced-order model for nearby trajectories. Modeling the dynamics of trajectories further away from the primary SSM, however, is difficult if the linear part of the system exhibits strong non-normal behavior. Such non-normality implies that simply projecting trajectories onto SSMs along directions normal to the slow linear modes will not pair those trajectories correctly with their reduced counterparts on the SSMs. In principle, a well-defined nonlinear projection along a stable invariant foliation exists and would exactly match the full dynamics to the SSM-reduced dynamics. This foliation, however, cannot realistically be constructed from practically feasible amounts and distributions of experimental data. Here, we develop an oblique projection technique that is able to approximate this foliation efficiently, even from a single experimental trajectory of a significantly non-normal and nonlinear beam.

Time-weighted kernel density for gearbox residual life prediction

With improvements in industrial automation, the reliability of the gearbox, a key transmission device, has become increasingly crucial for the stable operation of an entire operating system. However, predicting the remaining useful life of the gearbox is challenging because of complex working environments and dynamic load changes. Several existing methods assume an inaccurate model structure and parameter estimation during life prediction, owing to the limited availability of similar fault sample data. In this study, we analyse the influence of kernel density estimation (KDE) based on time-varying distribution on the results of residual useful life prediction, considering the characteristics of such systems and the problems faced by current research methods. First, a time-varying KDE model with an incremental distribution of degradation features is established, and the influence of sample timing on KDE is introduced. Second, the exponential weighted moving average method is employed to predict the degraded samples, and recursive update was employed to reduce unnecessary double calculations during the estimation of the time-varying weight kernel density in the system operation process. Finally, the adaptability and effectiveness of the proposed method are verified using actual collected gearbox data. Research results indicate that the remaining useful life prediction outcomes of the method proposed in this paper are superior to those of the DGN model and the Ensemble model, as evidenced by its lower RMSE and MAE values.

Strong and weak prediction of stochastic dynamics using reservoir computing

We propose an approach to replicate a stochastic system and forecast its dynamics using a reservoir computing (RC). We show that such machine learning models enable the prediction of the behavior of stochastic systems in a wide range of control parameters. However, the quality of forecasting depends significantly on the training approach used for the RC. Specifically, we distinguish two types of prediction-weak and strong predictions. We get what is called a strong prediction when the testing parameters are close to the training parameters, and almost a true replica of the system trajectory is obtained, which is determined by noise and initial conditions. On the contrary, we call the prediction weak if we can only predict probabilistic characteristics of a stochastic process, which happens if there exists a mismatch between training and testing parameters. The efficiency of our approach is demonstrated with the models of single and coupled stochastic FitzHugh-Nagumo oscillators and the model of an erbium-doped fiber laser with noisy diode pumping. With the help of a RC, we predict the system dynamics for a wide range of noise parameters. In addition, we find a particular regime when the model exhibits switches between strong and weak prediction types, resembling probabilistic properties of on-off intermittency.

Enhancing strategic decision-making in differential games through bifurcation prediction

Qualitative changes can occur in the dynamics of nonlinear systems even for small parameter variations. Such changes are manifestations of bifurcation in dynamical systems. In the context of differential game theory, bifurcations offer insights into the underlying mechanisms driving strategic interactions and identify transitions between different types of behavior. Such critical transitions are tipping points that can dramatically change the outcomes of the game. This work explores the possibility of predicting such qualitative shifts, including supercritical Hopf bifurcations, before they occur using a data-driven forecasting technique. This concept is demonstrated for an attacker-defender game in a limited resource scenario and for an active cybersecurity defense game. The time histories of the system dynamics as it approaches a bifurcation allow one player to detect the existence of bifurcations. This capability provides that player insights into the dynamics of the game and potential defense mechanisms in resource-constrained scenarios.

A mathematical framework for evo-devo dynamics

Natural selection acts on phenotypes constructed over development, which raises the question of how development affects evolution. Classic evolutionary theory indicates that development affects evolution by modulating the genetic covariation upon which selection acts, thus affecting genetic constraints. However, whether genetic constraints are relative, thus diverting adaptation from the direction of steepest fitness ascent, or absolute, thus blocking adaptation in certain directions, remains uncertain. This limits understanding of long-term evolution of developmentally constructed phenotypes. Here we formulate a general, tractable mathematical framework that integrates age progression, explicit development (i.e., the construction of the phenotype across life subject to developmental constraints), and evolutionary dynamics, thus describing the evolutionary and developmental (evo-devo) dynamics. The framework yields simple equations that can be arranged in a layered structure that we call the evo-devo process, whereby five core elementary components generate all equations including those mechanistically describing genetic covariation and the evo-devo dynamics. The framework recovers evolutionary dynamic equations in gradient form and describes the evolution of genetic covariation from the evolution of genotype, phenotype, environment, and mutational covariation. This shows that genotypic and phenotypic evolution must be followed simultaneously to yield a dynamically sufficient description of long-term phenotypic evolution in gradient form, such that evolution described as the climbing of a fitness landscape occurs in "geno-phenotype" space. Genetic constraints in geno-phenotype space are necessarily absolute because the phenotype is related to the genotype by development. Thus, the long-term evolutionary dynamics of developed phenotypes is strongly non-standard: (1) evolutionary equilibria are either absent or infinite in number and depend on genetic covariation and hence on development; (2) developmental constraints determine the admissible evolutionary path and hence which evolutionary equilibria are admissible; and (3) evolutionary outcomes occur at admissible evolutionary equilibria, which do not generally occur at fitness landscape peaks in geno-phenotype space, but at peaks in the admissible evolutionary path where "total genotypic selection" vanishes if exogenous plastic response vanishes and mutational variation exists in all directions of genotype space. Hence, selection and development jointly define the evolutionary outcomes if absolute mutational constraints and exogenous plastic response are absent, rather than the outcomes being defined only by selection. Moreover, our framework provides formulas for the sensitivities of a recurrence and an alternative method to dynamic optimization (i.e., dynamic programming or optimal control) to identify evolutionary outcomes in models with developmentally dynamic traits. These results show that development has major evolutionary effects.

Learning nonlinear projections for reduced-order modeling of dynamical systems using constrained autoencoders

Recently developed reduced-order modeling techniques aim to approximate nonlinear dynamical systems on low-dimensional manifolds learned from data. This is an effective approach for modeling dynamics in a post-transient regime where the effects of initial conditions and other disturbances have decayed. However, modeling transient dynamics near an underlying manifold, as needed for real-time control and forecasting applications, is complicated by the effects of fast dynamics and nonnormal sensitivity mechanisms. To begin to address these issues, we introduce a parametric class of nonlinear projections described by constrained autoencoder neural networks in which both the manifold and the projection fibers are learned from data. Our architecture uses invertible activation functions and biorthogonal weight matrices to ensure that the encoder is a left inverse of the decoder. We also introduce new dynamics-aware cost functions that promote learning of oblique projection fibers that account for fast dynamics and nonnormality. To demonstrate these methods and the specific challenges they address, we provide a detailed case study of a three-state model of vortex shedding in the wake of a bluff body immersed in a fluid, which has a two-dimensional slow manifold that can be computed analytically. In anticipation of future applications to high-dimensional systems, we also propose several techniques for constructing computationally efficient reduced-order models using our proposed nonlinear projection framework. This includes a novel sparsity-promoting penalty for the encoder that avoids detrimental weight matrix shrinkage via computation on the Grassmann manifold.

Distilling identifiable and interpretable dynamic models from biological data

Mechanistic dynamical models allow us to study the behavior of complex biological systems. They can provide an objective and quantitative understanding that would be difficult to achieve through other means. However, the systematic development of these models is a non-trivial exercise and an open problem in computational biology. Currently, many research efforts are focused on model discovery, i.e. automating the development of interpretable models from data. One of the main frameworks is sparse regression, where the sparse identification of nonlinear dynamics (SINDy) algorithm and its variants have enjoyed great success. SINDy-PI is an extension which allows the discovery of rational nonlinear terms, thus enabling the identification of kinetic functions common in biochemical networks, such as Michaelis-Menten. SINDy-PI also pays special attention to the recovery of parsimonious models (Occam's razor). Here we focus on biological models composed of sets of deterministic nonlinear ordinary differential equations. We present a methodology that, combined with SINDy-PI, allows the automatic discovery of structurally identifiable and observable models which are also mechanistically interpretable. The lack of structural identifiability and observability makes it impossible to uniquely infer parameter and state variables, which can compromise the usefulness of a model by distorting its mechanistic significance and hampering its ability to produce biological insights. We illustrate the performance of our method with six case studies. We find that, despite enforcing sparsity, SINDy-PI sometimes yields models that are unidentifiable. In these cases we show how our method transforms their equations in order to obtain a structurally identifiable and observable model which is also interpretable.

Optimizing the combination of data-driven and model-based elements in hybrid reservoir computing

Hybrid reservoir computing combines purely data-driven machine learning predictions with a physical model to improve the forecasting of complex systems. In this study, we investigate in detail the predictive capabilities of three different architectures for hybrid reservoir computing: the input hybrid (IH), output hybrid (OH), and full hybrid (FH), which combines IH and OH. By using nine different three-dimensional chaotic model systems and the high-dimensional spatiotemporal chaotic Kuramoto-Sivashinsky system, we demonstrate that all hybrid reservoir computing approaches significantly improve the prediction results, provided that the model is sufficiently accurate. For accurate models, we find that the OH and FH results are equivalent and significantly outperform the IH results, especially for smaller reservoir sizes. For totally inaccurate models, the predictive capabilities of IH and FH may decrease drastically, while the OH architecture remains as accurate as the purely data-driven results. Furthermore, OH allows for the separation of the reservoir and the model contributions to the output predictions. This enables an interpretation of the roles played by the data-driven and model-based elements in output hybrid reservoir computing, resulting in higher explainability of the prediction results. Overall, our findings suggest that the OH approach is the most favorable architecture for hybrid reservoir computing, when taking accuracy, interpretability, robustness to model error, and simplicity into account.

Nonlinear model reduction to fractional and mixed-mode spectral submanifolds

A primary spectral submanifold (SSM) is the unique smoothest nonlinear continuation of a nonresonant spectral subspace E of a dynamical system linearized at a fixed point. Passing from the full nonlinear dynamics to the flow on an attracting primary SSM provides a mathematically precise reduction of the full system dynamics to a very low-dimensional, smooth model in polynomial form. A limitation of this model reduction approach has been, however, that the spectral subspace yielding the SSM must be spanned by eigenvectors of the same stability type. A further limitation has been that in some problems, the nonlinear behavior of interest may be far away from the smoothest nonlinear continuation of the invariant subspace E. Here, we remove both of these limitations by constructing a significantly extended class of SSMs that also contains invariant manifolds with mixed internal stability types and of lower smoothness class arising from fractional powers in their parametrization. We show on examples how fractional and mixed-mode SSMs extend the power of data-driven SSM reduction to transitions in shear flows, dynamic buckling of beams, and periodically forced nonlinear oscillatory systems. More generally, our results reveal the general function library that should be used beyond integer-powered polynomials in fitting nonlinear reduced-order models to data.

How development affects evolution

Natural selection acts on developmentally constructed phenotypes, but how does development affect evolution? This question prompts a simultaneous consideration of development and evolution. However, there has been a lack of general mathematical frameworks mechanistically integrating the two, which may have inhibited progress on the question. Here, we use a new mathematical framework that mechanistically integrates development into evolution to analyse how development affects evolution. We show that, while selection pushes genotypic and phenotypic evolution up the fitness landscape, development determines the admissible evolutionary pathway, such that evolutionary outcomes occur at path peaks rather than landscape peaks. Changes in development can generate path peaks, triggering genotypic or phenotypic diversification, even on constant, single-peak landscapes. Phenotypic plasticity, niche construction, extra-genetic inheritance, and developmental bias alter the evolutionary path and hence the outcome. Thus, extra-genetic inheritance can have permanent evolutionary effects by changing the developmental constraints, even if extra-genetically acquired elements are not transmitted to future generations. Selective development, whereby phenotype construction points in the adaptive direction, may induce adaptive or maladaptive evolution depending on the developmental constraints. Moreover, developmental propagation of phenotypic effects over age enables the evolution of negative senescence. Overall, we find that development plays a major evolutionary role.