Information theory explanation of the fluctuation theorem, maximum entropy production and self-organized criticality in non-equilibrium stationary states

Type of probability distribution reflects how close mixing dynamics in river chemistry are to thermodynamic equilibrium

The distribution of geochemical species are typically either (log)normally distributed or follow power laws. Here we link these types of distributions to the dynamics of the system that generates these distributions, showing that power laws can emerge in dissipative systems far from equilibrium while (log)normal distributions are found for species for which the concentrations are close to equilibrium. We use observations of the chemical composition of river water from the sampling space in central Italy as well as discharge data to test this interpretation. We estimate the dissipation rate that results when groundwater drains into the river and the dissolved chemical species mix with the river water. We show that calcium (Ca2+) and bicarbonate (HCO3-) concentrations are close to saturation along most of the downstream length of the Arno river, with decreasing dissipation rates and a lognormal distribution, while sodium (Na+) and chloride (Cl-) concentrations increase substantially downstream, show increased dissipation rates, and are power-law distributed. This supports our hypothesis that power law distributions appear to be indicative of dissipative systems far from thermodynamic equilibrium, while (log)normal distributions indicate weakly dissipative systems close to equilibrium. What this implies is that probability distributions are likely to be indicative of the thermodynamics of the system and the magnitude of disequilibrium constrains the range over which power-law scaling may be observed. This should help us to better identify the generalities and mechanisms that result in these common types of distributions and to better classify variability in systems according to how dissipative these are.

Planetary Energy Flow and Entropy Production Rate by Earth from 2002 to 2023

In this work, satellite data from the Clouds and Earth's Radiant Energy System (CERES) and Moderate Resolution Imaging Spectroradiometer (MODIS) instruments are analyzed to determine how the global absorbed sunlight and global entropy production rates have changed from 2002 to 2023. The data is used to test hypotheses derived from the Maximum Power Principle (MPP) and Maximum Entropy Production Principle (MEP) about the evolution of Earth's surface and atmosphere. The results indicate that both the rate of absorbed sunlight and global entropy production have increased over the last 20 years, which is consistent with the predictions of both hypotheses. Given the acceptance of the MPP or MEP, some peripheral extensions and nuances are discussed.

Analyzing entropy features in time-series data for pattern recognition in neurological conditions

In the field of medical diagnosis and patient monitoring, effective pattern recognition in neurological time-series data is essential. Traditional methods predominantly based on statistical or probabilistic learning and inference often struggle with multivariate, multi-source, state-varying, and noisy data while also posing privacy risks due to excessive information collection and modeling. Furthermore, these methods often overlook critical statistical information, such as the distribution of data points and inherent uncertainties. To address these challenges, we introduce an information theory-based pipeline that leverages specialized features to identify patterns in neurological time-series data while minimizing privacy risks. We incorporate various entropy methods based on the characteristics of different scenarios and entropy. For stochastic state transition applications, we incorporate Shannon's entropy, entropy rates, entropy production, and the von Neumann entropy of Markov chains. When state modeling is impractical, we select and employ approximate entropy, increment entropy, dispersion entropy, phase entropy, and slope entropy. The pipeline's effectiveness and scalability are demonstrated through pattern analysis in a dementia care dataset and also an epileptic and a myocardial infarction dataset. The results indicate that our information theory-based pipeline can achieve average performance improvements across various models on the recall rate, F1 score, and accuracy by up to 13.08 percentage points, while enhancing inference efficiency by reducing the number of model parameters by an average of 3.10 times. Thus, our approach opens a promising avenue for improved, efficient, and critical statistical information-considered pattern recognition in medical time-series data.

Evaluating the Adiabatic Invariants in Magnetized Plasmas Using a Classical Ehrenfest Theorem

In this article, we address the reliance on probability density functions to obtain macroscopic properties in systems with multiple degrees of freedom as plasmas, and the limitations of expensive techniques for solving Equations such as Vlasov's. We introduce the Ehrenfest procedure as an alternative tool that promises to address these challenges more efficiently. Based on the conjugate variable theorem and the well-known fluctuation-dissipation theorem, this procedure offers a less expensive way of deriving time evolution Equations for macroscopic properties in systems far from equilibrium. We investigate the application of the Ehrenfest procedure for the study of adiabatic invariants in magnetized plasmas. We consider charged particles trapped in a dipole magnetic field and apply the procedure to the study of adiabatic invariants in magnetized plasmas and derive Equations for the magnetic moment, longitudinal invariant, and magnetic flux. We validate our theoretical predictions using a test particle simulation, showing good agreement between theory and numerical results for these observables. Although we observed small differences due to time scales and simulation limitations, our research supports the utility of the Ehrenfest procedure for understanding and modeling the behavior of particles in magnetized plasmas. We conclude that this procedure provides a powerful tool for the study of dynamical systems and statistical mechanics out of equilibrium, and opens perspectives for applications in other systems with probabilistic continuity.

Is Darwinian selection a retrograde driving force of evolution?

Modern science has still not provided a satisfactory empirical explanation for the increasing complexity of living organisms through evolutionary history. As no agreed-upon definitions of the complexity exist, the working definition of biological complexity has been formulated. There is no theoretical reason to expect evolutionary lineages to increase in complexity over time, and there is no empirical evidence that they do so. In our discussion we have assumed the hypothesis that at the origins of life, evolution had to first involve autocatalytic systems that only subsequently acquired the capacity of genetic heredity. We discuss the role of Darwinian selection in evolution and pose the hypothesis that Darwinian selection acts predominantly as a retrograde driving force of evolution. In this context we understand the term retrograde evolution as a degeneration of living systems from higher complexity towards living systems with lower complexity. With the proposed hypothesis we have closed the gap between Darwinism and Lamarckism early in the evolutionary process. By Lamarckism, the action of a special principle called complexification force is understood here rather than inheritance of acquired characteristics.

Chiral conformity emerges from the least-time free energy consumption

The prevalence of chirally pure biological polymers is often assumed to stem from some slight preference for one chiral form at the origin of life. Likewise, the predominance of matter over antimatter is presumed to follow from some subtle bias for matter at the dawn of the universe. However, rather than being imposed from the start, handedness standards in societies emerged to make things work. Since work is the universal measure of transferred energy, it is reasoned that standards at all scales and scopes emerge to consume free energy. Free energy minimization, equal to entropy maximization, turns out to be the second law of thermodynamics when derived from statistical physics of open systems. This many-body theory is based on the atomistic axiom that everything comprises the same fundamental elements known as quanta of action; hence, everything follows the same law. According to the thermodynamic principle, the flows of energy naturally select standard structures over less-fit functional forms to consume free energy in the least time. Thermodynamics making no distinction between animate and inanimate renders the question of life's handedness meaningless and deems the search for an intrinsic difference between matter and antimatter pointless.

Improvement of Error Correction in Nonequilibrium Information Dynamics

Errors are inevitable in information processing and transfer. While error correction is widely studied in engineering, the underlying physics is not fully understood. Due to the complexity and energy exchange involved, information transmission should be considered as a nonequilibrium process. In this study, we investigate the effects of nonequilibrium dynamics on error correction using a memoryless channel model. Our findings suggest that error correction improves as nonequilibrium increases, and the thermodynamic cost can be utilized to improve the correction quality. Our results inspire new approaches to error correction that incorporate nonequilibrium dynamics and thermodynamics, and highlight the importance of the nonequilibrium effects in error correction design, particularly in biological systems.

Self-Organization of Enzyme-Catalyzed Reactions Studied by the Maximum Entropy Production Principle

The self-organization of open reaction systems is closely related to specific mechanisms that allow the export of internally generated entropy from systems to their environment. According to the second law of thermodynamics, systems with effective entropy export to the environment are better internally organized. Therefore, they are in thermodynamic states with low entropy. In this context, we study how self-organization in enzymatic reactions depends on their kinetic reaction mechanisms. Enzymatic reactions in an open system are considered to operate in a non-equilibrium steady state, which is achieved by satisfying the principle of maximum entropy production (MEPP). The latter is a general theoretical framework for our theoretical analysis. Detailed theoretical studies and comparisons of the linear irreversible kinetic schemes of an enzyme reaction in two and three states are performed. In both cases, in the optimal and statistically most probable thermodynamic steady state, a diffusion-limited flux is predicted by MEPP. Several thermodynamic quantities and enzymatic kinetic parameters, such as the entropy production rate, the Shannon information entropy, reaction stability, sensitivity, and specificity constants, are predicted. Our results show that the optimal enzyme performance may strongly depend on the number of reaction steps when linear reaction mechanisms are considered. Simple reaction mechanisms with a smaller number of intermediate reaction steps could be better organized internally and could allow fast and stable catalysis. These could be features of the evolutionary mechanisms of highly specialized enzymes.

A broad perspective to particle-laden fluid interfaces systems: from chemically homogeneous particles to active colloids

Particles adsorbed to fluid interfaces are ubiquitous in industry, nature or life. The wide range of properties arising from the assembly of particles at fluid interface has stimulated an intense research activity on shed light to the most fundamental physico-chemical aspects of these systems. These include the mechanisms driving the equilibration of the interfacial layers, trapping energy, specific inter-particle interactions and the response of the particle-laden interface to mechanical perturbations and flows. The understanding of the physico-chemistry of particle-laden interfaces becomes essential for taking advantage of the particle capacity to stabilize interfaces for the preparation of different dispersed systems (emulsions, foams or colloidosomes) and the fabrication of new reconfigurable interface-dominated devices. This review presents a detailed overview of the physico-chemical aspects that determine the behavior of particles trapped at fluid interfaces. This has been combined with some examples of real and potential applications of these systems in technological and industrial fields. It is expected that this information can provide a general perspective of the topic that can be exploited for researchers and technologist non-specialized in the study of particle-laden interfaces, or for experienced researcher seeking new questions to solve.

Statistical Physics through the Lens of Real-Space Mutual Information

Identifying the relevant degrees of freedom in a complex physical system is a key stage in developing effective theories in and out of equilibrium. The celebrated renormalization group provides a framework for this, but its practical execution in unfamiliar systems is fraught with ad hoc choices, whereas machine learning approaches, though promising, lack formal interpretability. Here we present an algorithm employing state-of-the-art results in machine-learning-based estimation of information-theoretic quantities, overcoming these challenges, and use this advance to develop a new paradigm in identifying the most relevant operators describing properties of the system. We demonstrate this on an interacting model, where the emergent degrees of freedom are qualitatively different from the microscopic constituents. Our results push the boundary of formally interpretable applications of machine learning, conceptually paving the way toward automated theory building.

Thermodynamic constraints on the assembly and diversity of microbial ecosystems are different near to and far from equilibrium

Non-equilibrium thermodynamics has long been an area of substantial interest to ecologists because most fundamental biological processes, such as protein synthesis and respiration, are inherently energy-consuming. However, most of this interest has focused on developing coarse ecosystem-level maximisation principles, providing little insight into underlying mechanisms that lead to such emergent constraints. Microbial communities are a natural system to decipher this mechanistic basis because their interactions in the form of substrate consumption, metabolite production, and cross-feeding can be described explicitly in thermodynamic terms. Previous work has considered how thermodynamic constraints impact competition between pairs of species, but restrained from analysing how this manifests in complex dynamical systems. To address this gap, we develop a thermodynamic microbial community model with fully reversible reaction kinetics, which allows direct consideration of free-energy dissipation. This also allows species to interact via products rather than just substrates, increasing the dynamical complexity, and allowing a more nuanced classification of interaction types to emerge. Using this model, we find that community diversity increases with substrate lability, because greater free-energy availability allows for faster generation of niches. Thus, more niches are generated in the time frame of community establishment, leading to higher final species diversity. We also find that allowing species to make use of near-to-equilibrium reactions increases diversity in a low free-energy regime. In such a regime, two new thermodynamic interaction types that we identify here reach comparable strengths to the conventional (competition and facilitation) types, emphasising the key role that thermodynamics plays in community dynamics. Our results suggest that accounting for realistic thermodynamic constraints is vital for understanding the dynamics of real-world microbial communities.

There is a growing interest in microbial communities due to their important role in biogeochemical cycling as well as plant and animal health. Although our understanding of thermodynamic constraints on individual cells is rapidly improving, the impact of these constraints on complex microbial communities remains largely unexplored theoretically and empirically. Here, we develop a new microbial community model which allows thermodynamic efficiency and entropy production to be calculated directly. We find that availability of substrates with greater free-energy allows for a faster rate of niche generation, leading to higher final species diversity. We also show that when the free-energy availability is low, species with reactions close to thermodynamic equilibrium are favoured, leading to more diverse and efficient communities. In addition to the conventional interaction types (competition and facilitation), our model reveals the existence of two novel interaction types mediated by products rather than substrates. Though the conventional interactions are generally the strongest, the novel interaction types are significant when free-energy availability is low. Our results suggest that non-equilibrium thermodynamics need to be considered when studying microbial community dynamics.

Information Geometry, Fluctuations, Non-Equilibrium Thermodynamics, and Geodesics in Complex Systems

Information theory provides an interdisciplinary method to understand important phenomena in many research fields ranging from astrophysical and laboratory fluids/plasmas to biological systems. In particular, information geometric theory enables us to envision the evolution of non-equilibrium processes in terms of a (dimensionless) distance by quantifying how information unfolds over time as a probability density function (PDF) evolves in time. Here, we discuss some recent developments in information geometric theory focusing on time-dependent dynamic aspects of non-equilibrium processes (e.g., time-varying mean value, time-varying variance, or temperature, etc.) and their thermodynamic and physical/biological implications. We compare different distances between two given PDFs and highlight the importance of a path-dependent distance for a time-dependent PDF. We then discuss the role of the information rate Γ=dLdt and relative entropy in non-equilibrium thermodynamic relations (entropy production rate, heat flux, dissipated work, non-equilibrium free energy, etc.), and various inequalities among them. Here, L is the information length representing the total number of statistically distinguishable states a PDF evolves through over time. We explore the implications of a geodesic solution in information geometry for self-organization and control.

A Relativistic Entropic Hamiltonian–Lagrangian Approach to the Entropy Production of Spiral Galaxies in Hyperbolic Spacetime

Double-spiral galaxies are common in the Universe. It is known that the logarithmic double spiral is a Maximum Entropy geometry in hyperbolic (flat) spacetime that well represents an idealised spiral galaxy, with its central supermassive black hole (SMBH) entropy accounting for key galactic structural features including the stability and the double-armed geometry. Over time the central black hole must accrete mass, with the overall galactic entropy increasing: the galaxy is not at equilibrium. From the associated entropic Euler–Lagrange Equation (enabling the application of Noether’s theorem) we develop analytic expressions for the galactic entropy production of an idealised spiral galaxy showing that it is a conserved quantity, and we also derive an appropriate expression for its relativistic entropic Hamiltonian. We generalise Onsager’s celebrated expression for entropy production and demonstrate that galactic entropy production (entropy production corresponds to the intrinsic dissipation characteristics) is composed of two parts, one many orders of magnitude larger than the other: the smaller is comparable to the Hawking radiation of the central SMBH, while the other is comparable to the high entropy processes occurring within the accretion disks of real SMBHs. We conclude that galaxies cannot be isolated, since even idealised spiral galaxies intrinsically have a non-zero entropy production.

Close to Optimal Cell Sensing Ensures the Robustness of Tissue Differentiation Process: The Avian Photoreceptor Mosaic Case

The way that progenitor cell fate decisions and the associated environmental sensing are regulated to ensure the robustness of the spatial and temporal order in which cells are generated towards a fully differentiating tissue still remains elusive. Here, we investigate how cells regulate their sensing intensity and radius to guarantee the required thermodynamic robustness of a differentiated tissue. In particular, we are interested in finding the conditions where dedifferentiation at cell level is possible (microscopic reversibility), but tissue maintains its spatial order and differentiation integrity (macroscopic irreversibility). In order to tackle this, we exploit the recently postulated Least microEnvironmental Uncertainty Principle (LEUP) to develop a theory of stochastic thermodynamics for cell differentiation. To assess the predictive and explanatory power of our theory, we challenge it against the avian photoreceptor mosaic data. By calibrating a single parameter, the LEUP can predict the cone color spatial distribution in the avian retina and, at the same time, suggest that such a spatial pattern is associated with quasi-optimal cell sensing. By means of the stochastic thermodynamics formalism, we find out that thermodynamic robustness of differentiated tissues depends on cell metabolism and cell sensing properties. In turn, we calculate the limits of the cell sensing radius that ensure the robustness of differentiated tissue spatial order. Finally, we further constrain our model predictions to the avian photoreceptor mosaic.

A Criterion for the Formation of Nonequilibrium Self-Assembled Structures

Is there a criterion able to determine the type of structures formed in a nonequilibrium self-assembly process? This important question has a clear answer when the process takes place under equilibrium conditions: structures emerge at minimum values of the free energy. Experiments, however, have shown that when self-assembly takes place outside equilibrium, they do not appear at those free energy minima but rather at optimal values of structural parameters. On the basis of these observations, we propose a selection criterion for which structures come up at the minima of a nonequilibrium free energy that takes into account the energy needed to change their configuration. The criterion is able to predict the formation and configuration of structures such as Liesegang rings and patterns in magnetic colloids and could constitute a powerful tool to understand the synthesis of advanced materials, enantiomers, and nanoparticles.

Flexibility of enzymatic transitions as a hallmark of optimized enzyme steady-state kinetics and thermodynamics

We investigate the relations between the enzyme kinetic flexibility, the rate of entropy production, and the Shannon information entropy in a steady-state enzyme reaction. All these quantities are maximized with respect to enzyme rate constants. We show that the steady-state, which is characterized by the most flexible enzymatic transitions between the enzyme conformational states, coincides with the global maxima of the Shannon information entropy and the rate of entropy production. This steady-state of an enzyme is referred to as globally optimal. This theoretical approach is then used for the analysis of the kinetic and the thermodynamic performance of the enzyme triose-phosphate isomerase. The analysis reveals that there exist well-defined maxima of the kinetic flexibility, the rate of entropy production, and the Shannon information entropy with respect to any arbitrarily chosen rate constant of the enzyme and that these maxima, calculated from the measured kinetic rate constants for the triose-phosphate isomerase are lower, however of the same order of magnitude, as the maxima of the globally optimal state of the enzyme. This suggests that the triose-phosphate isomerase could be a well, but not fully evolved enzyme, as it was previously claimed. Herein presented theoretical investigations also provide clear evidence that the flexibility of enzymatic transitions between the enzyme conformational states is a requirement for the maximal Shannon information entropy and the maximal rate of entropy production.

Phytoplankton Temporal Strategies Increase Entropy Production in a Marine Food Web Model

We develop a trait-based model founded on the hypothesis that biological systems evolve and organize to maximize entropy production by dissipating chemical and electromagnetic free energy over longer time scales than abiotic processes by implementing temporal strategies. A marine food web consisting of phytoplankton, bacteria, and consumer functional groups is used to explore how temporal strategies, or the lack thereof, change entropy production in a shallow pond that receives a continuous flow of reduced organic carbon plus inorganic nitrogen and illumination from solar radiation with diel and seasonal dynamics. Results show that a temporal strategy that employs an explicit circadian clock produces more entropy than a passive strategy that uses internal carbon storage or a balanced growth strategy that requires phytoplankton to grow with fixed stoichiometry. When the community is forced to operate at high specific growth rates near 2 d⁻¹, the optimization-guided model selects for phytoplankton ecotypes that exhibit complementary for winter versus summer environmental conditions to increase entropy production. We also present a new type of trait-based modeling where trait values are determined by maximizing entropy production rather than by random selection.

Enzyme activities predicted by metabolite concentrations and solvent capacity in the cell

Experimental measurements or computational model predictions of the post-translational regulation of enzymes needed in a metabolic pathway is a difficult problem. Consequently, regulation is mostly known only for well-studied reactions of central metabolism in various model organisms. In this study, we use two approaches to predict enzyme regulation policies and investigate the hypothesis that regulation is driven by the need to maintain the solvent capacity in the cell. The first predictive method uses a statistical thermodynamics and metabolic control theory framework while the second method is performed using a hybrid optimization-reinforcement learning approach. Efficient regulation schemes were learned from experimental data that either agree with theoretical calculations or result in a higher cell fitness using maximum useful work as a metric. As previously hypothesized, regulation is herein shown to control the concentrations of both immediate and downstream product concentrations at physiological levels. Model predictions provide the following two novel general principles: (1) the regulation itself causes the reactions to be much further from equilibrium instead of the common assumption that highly non-equilibrium reactions are the targets for regulation; and (2) the minimal regulation needed to maintain metabolite levels at physiological concentrations maximizes the free energy dissipation rate instead of preserving a specific energy charge. The resulting energy dissipation rate is an emergent property of regulation which may be represented by a high value of the adenylate energy charge. In addition, the predictions demonstrate that the amount of regulation needed can be minimized if it is applied at the beginning or branch point of a pathway, in agreement with common notions. The approach is demonstrated for three pathways in the central metabolism of E. coli (gluconeogenesis, glycolysis-tricarboxylic acid (TCA) and pentose phosphate-TCA) that each require different regulation schemes. It is shown quantitatively that hexokinase, glucose 6-phosphate dehydrogenase and glyceraldehyde phosphate dehydrogenase, all branch points of pathways, play the largest roles in regulating central metabolism.

An Introduction to the Non-Equilibrium Steady States of Maximum Entropy Spike Trains

Although most biological processes are characterized by a strong temporal asymmetry, several popular mathematical models neglect this issue. Maximum entropy methods provide a principled way of addressing time irreversibility, which leverages powerful results and ideas from the literature of non-equilibrium statistical mechanics. This tutorial provides a comprehensive overview of these issues, with a focus in the case of spike train statistics. We provide a detailed account of the mathematical foundations and work out examples to illustrate the key concepts and results from non-equilibrium statistical mechanics.

Maximum Entropy Analysis of Flow Networks: Theoretical Foundation and Applications

The concept of a “flow network”—a set of nodes and links which carries one or more flows—unites many different disciplines, including pipe flow, fluid flow, electrical, chemical reaction, ecological, epidemiological, neurological, communications, transportation, financial, economic and human social networks. This Feature Paper presents a generalized maximum entropy framework to infer the state of a flow network, including its flow rates and other properties, in probabilistic form. In this method, the network uncertainty is represented by a joint probability function over its unknowns, subject to all that is known. This gives a relative entropy function which is maximized, subject to the constraints, to determine the most probable or most representative state of the network. The constraints can include “observable” constraints on various parameters, “physical” constraints such as conservation laws and frictional properties, and “graphical” constraints arising from uncertainty in the network structure itself. Since the method is probabilistic, it enables the prediction of network properties when there is insufficient information to obtain a deterministic solution. The derived framework can incorporate nonlinear constraints or nonlinear interdependencies between variables, at the cost of requiring numerical solution. The theoretical foundations of the method are first presented, followed by its application to a variety of flow networks.

Maximum Entropy Production Theorem for Transitions between Enzyme Functional States and Its Applications

Transitions between enzyme functional states are often connected to conformational changes involving electron or proton transport and directional movements of a group of atoms. These microscopic fluxes, resulting in entropy production, are driven by non-equilibrium concentrations of substrates and products. Maximal entropy production exists for any chosen transition, but such a maximal transitional entropy production (MTEP) requirement does not ensure an increase of total entropy production, nor an increase in catalytic performance. We examine when total entropy production increases, together with an increase in the performance of an enzyme or bioenergetic system. The applications of the MTEP theorem for transitions between functional states are described for the triosephosphate isomerase, ATP synthase, for β-lactamases, and for the photochemical cycle of bacteriorhodopsin. The rate-limiting steps can be easily identified as those which are the most efficient in dissipating free-energy gradients and in performing catalysis. The last step in the catalytic cycle is usually associated with the highest free-energy dissipation involving proton nanocurents. This recovery rate-limiting step can be optimized for higher efficiency by using corresponding MTEP requirements. We conclude that biological evolution, leading to increased optimal catalytic efficiency, also accelerated the thermodynamic evolution, the synergistic relationship we named the evolution-coupling hypothesis.

Time-Energy and Time-Entropy Uncertainty Relations in Nonequilibrium Quantum Thermodynamics under Steepest-Entropy-Ascent Nonlinear Master Equations

In the domain of nondissipative unitary Hamiltonian dynamics, the well-known Mandelstam–Tamm–Messiah time–energy uncertainty relation τFΔH≥ℏ/2 provides a general lower bound to the characteristic time τF=ΔF/|d〈F〉/dt| with which the mean value of a generic quantum observable F can change with respect to the width ΔF of its uncertainty distribution (square root of F fluctuations). A useful practical consequence is that in unitary dynamics the states with longer lifetimes are those with smaller energy uncertainty ΔH (square root of energy fluctuations). Here we show that when unitary evolution is complemented with a steepest-entropy-ascent model of dissipation, the resulting nonlinear master equation entails that these lower bounds get modified and depend also on the entropy uncertainty ΔS (square root of entropy fluctuations). For example, we obtain the time–energy-and–time–entropy uncertainty relation (2τFΔH/ℏ)2+(τFΔS/kBτ)2≥1 where τ is a characteristic dissipation time functional that for each given state defines the strength of the nonunitary, steepest-entropy-ascent part of the assumed master equation. For purely dissipative dynamics this reduces to the time–entropy uncertainty relation τFΔS≥kBτ, meaning that the nonequilibrium dissipative states with longer lifetime are those with smaller entropy uncertainty ΔS.

Entropy Production, Entropy Generation, and Fokker-Planck Equations for Cancer Cell Growth

It is rather difficult to understand biological systems from a physics point of view, and understanding systems such as cancer is even more challenging. There are many factors affecting the dynamics of a cancer cell, and they can be understood approximately. We can apply the principles of non-equilibrium statistical mechanics and thermodynamics to have a greater understanding of such systems. Very much like other systems, living systems also transform energy and matter during metabolism, and according to the First Law of Thermodynamics, this could be described as a capacity to transform energy in a controlled way. The properties of cancer cells are different from regular cells. Cancer is a name used for a set of malignant cells that lost control over normal growth. Cancer can be described as an open, complex, dynamic, and self-organizing system. Cancer is considered as a non-linear dynamic system, which can be explained to a good degree using techniques from non-equilibrium statistical mechanics and thermodynamics. We will also look at such a system through its entropy due to to the interaction with the environment and within the system itself. Here, we have studied the entropy generation versus the entropy production approach, and have calculated the entropy of growth of cancer cells using Fokker-Planck equations.

Convective flow in the presence of a small obstacle: Symmetry breaking, attractors, hysteresis, and information

This work explores the stability and hysteresis effects that occur when a small sink of momentum is introduced into a heat-driven, two-dimensional convective flow. As per standard fluid mechanical intuition, the system minimizes work generation and dissipation when one component of momentum is extracted. However, when the sink absorbs all incoming momentum, the system configures itself such that one of the convection plumes aligns directly with the sink. This state is the most hydrodynamically stable, but it maximizes, rather than minimizes extracted mechanical work. Furthermore, in the case of only vertical momentum extraction, there are two attractors, with different stabilities. Numerical experiments involving slow variations of the horizontal momentum extraction show a clear history dependence. This hysteresis preserves information about the system's past states, and hence represents a primitive memory. The momentum sink can also be used to manipulate the horizontal position of the flow field, with potential applications in microfluidics and laminar convection systems. This simple system exhibits the phenomena of autocatalysis (during the initial growth of the convection plumes), negative feedback (the attractors are either fully or quasistable), memory, and elementary computation.

Self-assembling outside equilibrium: emergence of structures mediated by dissipation

Self-assembly under non-equilibrium conditions may give rise to the formation of structures not available at equilibrium.

Entropy Density Acceleration and Minimum Dissipation Principle: Correlation with Heat and Matter Transfer in Glucose Catabolism

The heat and matter transfer during glucose catabolism in living systems and their relation with entropy production are a challenging subject of the classical thermodynamics applied to biology. In this respect, an analogy between mechanics and thermodynamics has been performed via the definition of the entropy density acceleration expressed by the time derivative of the rate of entropy density and related to heat and matter transfer in minimum living systems. Cells are regarded as open thermodynamic systems that exchange heat and matter resulting from irreversible processes with the intercellular environment. Prigogine's minimum energy dissipation principle is reformulated using the notion of entropy density acceleration applied to glucose catabolism. It is shown that, for out-of-equilibrium states, the calculated entropy density acceleration for a single cell is finite and negative and approaches as a function of time a zero value at global thermodynamic equilibrium for heat and matter transfer independently of the cell type and the metabolic pathway. These results could be important for a deeper understanding of entropy generation and its correlation with heat transfer in cell biology with special regard to glucose catabolism representing the prototype of irreversible reactions and a crucial metabolic pathway in stem cells and cancer stem cells.

A Brief Review of Generalized Entropies

Entropy appears in many contexts (thermodynamics, statistical mechanics, information theory, measure-preserving dynamical systems, topological dynamics, etc.) as a measure of different properties (energy that cannot produce work, disorder, uncertainty, randomness, complexity, etc.). In this review, we focus on the so-called generalized entropies, which from a mathematical point of view are nonnegative functions defined on probability distributions that satisfy the first three Shannon–Khinchin axioms: continuity, maximality and expansibility. While these three axioms are expected to be satisfied by all macroscopic physical systems, the fourth axiom (separability or strong additivity) is in general violated by non-ergodic systems with long range forces, this having been the main reason for exploring weaker axiomatic settings. Currently, non-additive generalized entropies are being used also to study new phenomena in complex dynamics (multifractality), quantum systems (entanglement), soft sciences, and more. Besides going through the axiomatic framework, we review the characterization of generalized entropies via two scaling exponents introduced by Hanel and Thurner. In turn, the first of these exponents is related to the diffusion scaling exponent of diffusion processes, as we also discuss. Applications are addressed as the description of the main generalized entropies advances.

Fluctuation analysis of the atmospheric energy cycle

The atmosphere gains available potential energy by solar radiation and dissipates kinetic energy mainly in the atmospheric boundary layer. We analyze the fluctuations of the global mean energy cycle defined by Lorenz in a simulation with a simplified hydrostatic model. The energy current densities are well approximated by the generalized Gumbel distribution and the Generalized Extreme Value (GEV) distribution. In an attempt to assess the fluctuation relation of Evans, Cohen, and Morriss we define entropy production by the injected power and use the GEV location parameter as a reference state. The fluctuation ratio reveals a linear behavior in a finite range.

Probabilistic Inference for Dynamical Systems

A general framework for inference in dynamical systems is described, based on the language of Bayesian probability theory and making use of the maximum entropy principle. Taking the concept of a path as fundamental, the continuity equation and Cauchy's equation for fluid dynamics arise naturally, while the specific information about the system can be included using the maximum caliber (or maximum path entropy) principle.

Prediction of Metabolite Concentrations, Rate Constants and Post-Translational Regulation Using Maximum Entropy-Based Simulations with Application to Central Metabolism of Neurospora crassa

We report the application of a recently proposed approach for modeling biological systems using a maximum entropy production rate principle in lieu of having in vivo rate constants. The method is applied in four steps: (1) a new ordinary differential equation (ODE) based optimization approach based on Marcelin’s 1910 mass action equation is used to obtain the maximum entropy distribution; (2) the predicted metabolite concentrations are compared to those generally expected from experiments using a loss function from which post-translational regulation of enzymes is inferred; (3) the system is re-optimized with the inferred regulation from which rate constants are determined from the metabolite concentrations and reaction fluxes; and finally (4) a full ODE-based, mass action simulation with rate parameters and allosteric regulation is obtained. From the last step, the power characteristics and resistance of each reaction can be determined. The method is applied to the central metabolism of Neurospora crassa and the flow of material through the three competing pathways of upper glycolysis, the non-oxidative pentose phosphate pathway, and the oxidative pentose phosphate pathway are evaluated as a function of the NADP/NADPH ratio. It is predicted that regulation of phosphofructokinase (PFK) and flow through the pentose phosphate pathway are essential for preventing an extreme level of fructose 1,6-bisphophate accumulation. Such an extreme level of fructose 1,6-bisphophate would otherwise result in a glassy cytoplasm with limited diffusion, dramatically decreasing the entropy and energy production rate and, consequently, biological competitiveness.

Entrance and escape dynamics for the typical set

According to the asymptotic equipartition property, sufficiently long sequences of random variables converge to a set that is typical. While the size and probability of this set are central to information theory and statistical mechanics, they can often only be estimated accurately in the asymptotic limit due to the exponential growth in possible sequences. Here we derive a time-inhomogeneous dynamics that constructs the properties of the typical set for all finite length sequences of independent and identically distributed random variables. These dynamics link the finite properties of the typical set to asymptotic results and allow the typical set to be applied to small and transient systems. The main result is a geometric mapping-the triangle map-relating sequences of random variables of length n to those of length n+1. We show that the number of points in this map needed to quantify the properties of the typical set grows linearly with sequence length, despite the exponential growth in the number of typical sequences. We illustrate the framework for the Bernoulli process and the Schlögl model for autocatalytic chemical reactions and demonstrate both the convergence to asymptotic limits and the ability to reproduce exact calculations.

Optimal information networks: Application for data-driven integrated health in populations

Development of composite indicators for integrated health in populations typically relies on a priori assumptions rather than model-free, data-driven evidence. Traditional variable selection processes tend not to consider relatedness and redundancy among variables, instead considering only individual correlations. In addition, a unified method for assessing integrated health statuses of populations is lacking, making systematic comparison among populations impossible. We propose the use of maximum entropy networks (MENets) that use transfer entropy to assess interrelatedness among selected variables considered for inclusion in a composite indicator. We also define optimal information networks (OINs) that are scale-invariant MENets, which use the information in constructed networks for optimal decision-making. Health outcome data from multiple cities in the United States are applied to this method to create a systemic health indicator, representing integrated health in a city.

Entropy production in systems with random transition rates close to equilibrium

We study the entropy production of systems out of equilibrium using random networks. We focus on systems with a finite number of states described by a master equation close to equilibrium. The dynamics are mapped into a network of states (nodes) connected by transition rates (links). Using this framework, we analyze the entropy production of ensembles of randomly generated networks owing to specific constraints (e.g., size or symmetries) and identify the most important parameters that determine its value. This analysis gives a null-model estimation for the entropy production that can be used for comparison with specific systems.

Computational Foundations of Natural Intelligence

New developments in AI and neuroscience are revitalizing the quest to understanding natural intelligence, offering insight about how to equip machines with human-like capabilities. This paper reviews some of the computational principles relevant for understanding natural intelligence and, ultimately, achieving strong AI. After reviewing basic principles, a variety of computational modeling approaches is discussed. Subsequently, I concentrate on the use of artificial neural networks as a framework for modeling cognitive processes. This paper ends by outlining some of the challenges that remain to fulfill the promise of machines that show human-like intelligence.

Entropy production selects nonequilibrium states in multistable systems

Far-from-equilibrium thermodynamics underpins the emergence of life, but how has been a long-outstanding puzzle. Best candidate theories based on the maximum entropy production principle could not be unequivocally proven, in part due to complicated physics, unintuitive stochastic thermodynamics, and the existence of alternative theories such as the minimum entropy production principle. Here, we use a simple, analytically solvable, one-dimensional bistable chemical system to demonstrate the validity of the maximum entropy production principle. To generalize to multistable stochastic system, we use the stochastic least-action principle to derive the entropy production and its role in the stability of nonequilibrium steady states. This shows that in a multistable system, all else being equal, the steady state with the highest entropy production is favored, with a number of implications for the evolution of biological, physical, and geological systems.

Entropy and optimality in river deltas

River deltas are critically important Earthscapes at the land–water interface, supporting dense populations and diverse ecosystems while also providing disproportionately large food and energy resources. Deltas exhibit complex channel networks that dictate how water, sediment, and nutrients are spread over the delta surface. By adapting concepts from information theory, we show that a range of field and numerically generated deltas obey an optimality principle that suggests that deltas self-organize to increase the diversity of sediment transport pathways across the delta channels to the shoreline. We suggest that optimal delta configurations are also more resilient because the same mechanism that diversifies the delivery of fluxes to the shoreline also enhances the dampening of possible perturbations.

The form and function of river deltas is intricately linked to the evolving structure of their channel networks, which controls how effectively deltas are nourished with sediments and nutrients. Understanding the coevolution of deltaic channels and their flux organization is crucial for guiding maintenance strategies of these highly stressed systems from a range of anthropogenic activities. To date, however, a unified theory explaining how deltas self-organize to distribute water and sediment up to the shoreline remains elusive. Here, we provide evidence for an optimality principle underlying the self-organized partition of fluxes in delta channel networks. By introducing a suitable nonlocal entropy rate (nER) and by analyzing field and simulated deltas, we suggest that delta networks achieve configurations that maximize the diversity of water and sediment flux delivery to the shoreline. We thus suggest that prograding deltas attain dynamically accessible optima of flux distributions on their channel network topologies, thus effectively decoupling evolutionary time scales of geomorphology and hydrology. When interpreted in terms of delta resilience, high nER configurations reflect an increased ability to withstand perturbations. However, the distributive mechanism responsible for both diversifying flux delivery to the shoreline and dampening possible perturbations might lead to catastrophic events when those perturbations exceed certain intensity thresholds.

Non-steady state mass action dynamics without rate constants: dynamics of coupled reactions using chemical potentials

Comprehensive and predictive simulation of coupled reaction networks has long been a goal of biology and other fields. Currently, metabolic network models that utilize enzyme mass action kinetics have predictive power but are limited in scope and application by the fact that the determination of enzyme rate constants is laborious and low throughput. We present a statistical thermodynamic formulation of the law of mass action for coupled reactions at both steady states and non-stationary states. The formulation uses chemical potentials instead of rate constants. When used to model deterministic systems, the method corresponds to a rescaling of the time dependent reactions in such a way that steady states can be reached on the same time scale but with significantly fewer computational steps. The relationships between reaction affinities, free energy changes and generalized detailed balance are central to the discussion. The significance for applications in systems biology are discussed as is the concept and assumption of maximum entropy production rate as a biological principle that links thermodynamics to natural selection.

Symmetries and nonequilibrium thermodynamics

Thermodynamic systems can be defined as composed by many identical interacting subsystems. Here it is shown how the dynamics of relaxation toward equilibrium of a thermodynamic system is closely related to the symmetry group of the Hamiltonian of the subsystems of which it is composed. The transitions between states driven by the interactions between identical subsystems correspond to elements of the root system associated to the symmetry group of their Hamiltonian. This imposes constraints on the relaxation dynamics of the complete thermodynamic system, which allow formulating its evolution toward equilibrium as a system of linear differential equations in which the variables are the thermodynamic forces of the system. The trajectory of a thermodynamic system in the space of thermodynamic forces corresponds to the negative gradient of a potential function, which depends on the symmetry group of the Hamiltonian of the individual interacting subsystems.

Energy conservation and maximal entropy production in enzyme reactions

A procedure for maximization of the density of entropy production in a single stationary two-step enzyme reaction is developed. Under the constraints of mass conservation, fixed equilibrium constant of a reaction and fixed products of forward and backward enzyme rate constants the existence of maximum in the density of entropy production is demonstrated. In the state with maximal density of entropy production the optimal enzyme rate constants, the stationary concentrations of the substrate and the product, the stationary product yield as well as the stationary reaction flux are calculated. The test, whether these calculated values of the reaction parameters are consistent with their corresponding measured values, is performed for the enzyme Glucose Isomerase. It is found that calculated and measured rate constants agree within an order of magnitude, whereas the calculated reaction flux and the product yield differ from their corresponding measured values for less than 20 % and 5 %, respectively. This indicates that the enzyme Glucose Isomerase, considered in a non-equilibrium stationary state, as found in experiments using the continuous stirred tank reactors, possibly operates close to the state with the maximum in the density of entropy production.

Is the catalytic activity of triosephosphate isomerase fully optimized? An investigation based on maximization of entropy production

Triosephosphate isomerase (TIM) is often described as a fully evolved housekeeping enzyme with near-maximal possible reaction rate. The assumption that an enzyme is perfectly evolved has not been easy to confirm or refute. In this paper, we use maximization of entropy production within known constraints to examine this assumption by calculating steady-state cyclic flux, corresponding entropy production, and catalytic activity in a reversible four-state scheme of TIM functional states. The maximal entropy production (MaxEP) requirement for any of the first three transitions between TIM functional states leads to decreased total entropy production. Only the MaxEP requirement for the product (R-glyceraldehyde-3-phosphate) release step led to a 30% increase in enzyme activity, specificity constant kcat/KM, and overall entropy production. The product release step, due to the TIM molecular machine working in the physiological direction of glycolysis, has not been identified before as the rate-limiting step by using irreversible thermodynamics. Together with structural studies, our results open the possibility for finding amino acid substitutions leading to an increased frequency of loop six opening and product release.

Maximum caliber inference and the stochastic Ising model

We investigate the maximum caliber variational principle as an inference algorithm used to predict dynamical properties of complex nonequilibrium, stationary, statistical systems in the presence of incomplete information. Specifically, we maximize the path entropy over discrete time step trajectories subject to normalization, stationarity, and detailed balance constraints together with a path-dependent dynamical information constraint reflecting a given average global behavior of the complex system. A general expression for the transition probability values associated with the stationary random Markov processes describing the nonequilibrium stationary system is computed. By virtue of our analysis, we uncover that a convenient choice of the dynamical information constraint together with a perturbative asymptotic expansion with respect to its corresponding Lagrange multiplier of the general expression for the transition probability leads to a formal overlap with the well-known Glauber hyperbolic tangent rule for the transition probability for the stochastic Ising model in the limit of very high temperatures of the heat reservoir.