Stochastic Thermodynamics of Learning

Autonomous learning of generative models with chemical reaction network ensembles

Can a micron-sized sack of interacting molecules autonomously learn an internal model of a complex and fluctuating environment? We draw insights from control theory, machine learning theory, chemical reaction network theory and statistical physics to develop a general architecture whereby a broad class of chemical systems can autonomously learn complex distributions. Our construction takes the form of a chemical implementation of machine learning's optimization workhorse: gradient descent on the relative entropy cost function, which we demonstrate can be viewed as a form of integral feedback control. We show how this method can be applied to optimize any detailed balanced chemical reaction network and that the construction is capable of using hidden units to learn complex distributions.

Thermodynamic computing via autonomous quantum thermal machines

We develop a physics-based model for classical computation based on autonomous quantum thermal machines. These machines consist of few interacting quantum bits (qubits) connected to several environments at different temperatures. Heat flows through the machine are here exploited for computing. The process starts by setting the temperatures of the environments according to the logical input. The machine evolves, eventually reaching a nonequilibrium steady state, from which the output of the computation can be determined via the temperature of an auxilliary finite-size reservoir. Such a machine, which we term a "thermodynamic neuron," can implement any linearly separable function, and we discuss explicitly the cases of NOT, 3-MAJORITY, and NOR gates. In turn, we show that a network of thermodynamic neurons can perform any desired function. We discuss the close connection between our model and artificial neurons (perceptrons) and argue that our model provides an alternative physics-based analog implementation of neural networks, and more generally a platform for thermodynamic computing.

Neural Network Layer Algebra: A Framework to Measure Capacity and Compression in Deep Learning

We present a new framework to measure the intrinsic properties of (deep) neural networks. While we focus on convolutional networks, our framework can be extrapolated to any network architecture. In particular, we evaluate two network properties, namely, capacity, which is related to expressivity, and compression, which is related to learnability. Both these properties depend only on the network structure and are independent of the network parameters. To this end, we propose two metrics: the first one, called layer complexity, captures the architectural complexity of any network layer; and, the second one, called layer intrinsic power, encodes how data are compressed along the network. The metrics are based on the concept of layer algebra, which is also introduced in this article. This concept is based on the idea that the global properties depend on the network topology, and the leaf nodes of any neural network can be approximated using local transfer functions, thereby allowing a simple computation of the global metrics. We show that our global complexity metric can be calculated and represented more conveniently than the widely used Vapnik-Chervonenkis (VC) dimension. We also compare the properties of various state-of-the-art architectures using our metrics and use the properties to analyze their accuracy on benchmark image classification datasets.

Machine learning at the mesoscale: A computation-dissipation bottleneck

The cost of information processing in physical systems calls for a trade-off between performance and energetic expenditure. Here we formulate and study a computation-dissipation bottleneck in mesoscopic systems used as input-output devices. Using both real data sets and synthetic tasks, we show how nonequilibrium leads to enhanced performance. Our framework sheds light on a crucial compromise between information compression, input-output computation and dynamic irreversibility induced by nonreciprocal interactions.

Heteroclinic networks for brain dynamics

Heteroclinic networks are a mathematical concept in dynamic systems theory that is suited to describe metastable states and switching events in brain dynamics. The framework is sensitive to external input and, at the same time, reproducible and robust against perturbations. Solutions of the corresponding differential equations are spatiotemporal patterns that are supposed to encode information both in space and time coordinates. We focus on the concept of winnerless competition as realized in generalized Lotka-Volterra equations and report on results for binding and chunking dynamics, synchronization on spatial grids, and entrainment to heteroclinic motion. We summarize proposals of how to design heteroclinic networks as desired in view of reproducing experimental observations from neuronal networks and discuss the subtle role of noise. The review is on a phenomenological level with possible applications to brain dynamics, while we refer to the literature for a rigorous mathematical treatment. We conclude with promising perspectives for future research.

Physical Grounds for Causal Perspectivalism

We ground the asymmetry of causal relations in the internal physical states of a special kind of open and irreversible physical system, a causal agent. A causal agent is an autonomous physical system, maintained in a steady state, far from thermal equilibrium, with special subsystems: sensors, actuators, and learning machines. Using feedback, the learning machine, driven purely by thermodynamic constraints, changes its internal states to learn probabilistic functional relations inherent in correlations between sensor and actuator records. We argue that these functional relations just are causal relations learned by the agent, and so such causal relations are simply relations between the internal physical states of a causal agent. We show that learning is driven by a thermodynamic principle: the error rate is minimised when the dissipated power is minimised. While the internal states of a causal agent are necessarily stochastic, the learned causal relations are shared by all machines with the same hardware embedded in the same environment. We argue that this dependence of causal relations on such 'hardware' is a novel demonstration of causal perspectivalism.

Intensive Urbanization, Urban Meteorology and Air Pollutants: Effects on the Temperature of a City in a Basin Geography

A qualitative study of thermal transfers is carried out from a record of measurements (time series) of meteorological variables (temperature, relative humidity and magnitude of wind speeds) and pollutants (PM₁₀, PM_2.5 and CO) in six localities located at different heights in the geographic basin of Santiago de Chile. The measurements were made in two periods, 2010-2013 and 2017-2020 (a total of 2,049,336 data), the last period coinciding with a process of intense urbanization, especially high-rise construction. The measurements, in the form of hourly time series, are analyzed on the one hand according to the theory of thermal conduction discretizing the differential equation of the temporal variation in the temperature and, on the other hand, through the theory of chaos that provides the entropies (S). Both procedures demonstrate, comparatively, that the last period of intense urbanization presents an increase in thermal transfers and temperature, which affects urban meteorology and makes it more complex. As shown by the chaotic analysis, there is a faster loss of information for the period 2017-2020. The consequences of the increase in temperature on human health and learning processes are studied.

Thermodynamic fluctuation theorems govern human sensorimotor learning

The application of thermodynamic reasoning in the study of learning systems has a long tradition. Recently, new tools relating perfect thermodynamic adaptation to the adaptation process have been developed. These results, known as fluctuation theorems, have been tested experimentally in several physical scenarios and, moreover, they have been shown to be valid under broad mathematical conditions. Hence, although not experimentally challenged yet, they are presumed to apply to learning systems as well. Here we address this challenge by testing the applicability of fluctuation theorems in learning systems, more specifically, in human sensorimotor learning. In particular, we relate adaptive movement trajectories in a changing visuomotor rotation task to fully adapted steady-state behavior of individual participants. We find that human adaptive behavior in our task is generally consistent with fluctuation theorem predictions and discuss the merits and limitations of the approach.

Jarzyski's Equality and Crooks' Fluctuation Theorem for General Markov Chains with Application to Decision-Making Systems

We define common thermodynamic concepts purely within the framework of general Markov chains and derive Jarzynski's equality and Crooks' fluctuation theorem in this setup. In particular, we regard the discrete-time case, which leads to an asymmetry in the definition of work that appears in the usual formulation of Crooks' fluctuation theorem. We show how this asymmetry can be avoided with an additional condition regarding the energy protocol. The general formulation in terms of Markov chains allows transferring the results to other application areas outside of physics. Here, we discuss how this framework can be applied in the context of decision-making. This involves the definition of the relevant quantities, the assumptions that need to be made for the different fluctuation theorems to hold, as well as the consideration of discrete trajectories instead of the continuous trajectories, which are relevant in physics.

Optimal Population Coding for Dynamic Input by Nonequilibrium Networks

The efficient coding hypothesis states that neural response should maximize its information about the external input. Theoretical studies focus on optimal response in single neuron and population code in networks with weak pairwise interactions. However, more biological settings with asymmetric connectivity and the encoding for dynamical stimuli have not been well-characterized. Here, we study the collective response in a kinetic Ising model that encodes the dynamic input. We apply gradient-based method and mean-field approximation to reconstruct networks given the neural code that encodes dynamic input patterns. We measure network asymmetry, decoding performance, and entropy production from networks that generate optimal population code. We analyze how stimulus correlation, time scale, and reliability of the network affect optimal encoding networks. Specifically, we find network dynamics altered by statistics of the dynamic input, identify stimulus encoding strategies, and show optimal effective temperature in the asymmetric networks. We further discuss how this approach connects to the Bayesian framework and continuous recurrent neural networks. Together, these results bridge concepts of nonequilibrium physics with the analyses of dynamics and coding in networks.

Adaptive behaviour and learning in slime moulds: the role of oscillations

The slime mould Physarum polycephalum, an aneural organism, uses information from previous experiences to adjust its behaviour, but the mechanisms by which this is accomplished remain unknown. This article examines the possible role of oscillations in learning and memory in slime moulds. Slime moulds share surprising similarities with the network of synaptic connections in animal brains. First, their topology derives from a network of interconnected, vein-like tubes in which signalling molecules are transported. Second, network motility, which generates slime mould behaviour, is driven by distinct oscillations that organize into spatio-temporal wave patterns. Likewise, neural activity in the brain is organized in a variety of oscillations characterized by different frequencies. Interestingly, the oscillating networks of slime moulds are not precursors of nervous systems but, rather, an alternative architecture. Here, we argue that comparable information-processing operations can be realized on different architectures sharing similar oscillatory properties. After describing learning abilities and oscillatory activities of P. polycephalum, we explore the relation between network oscillations and learning, and evaluate the organism's global architecture with respect to information-processing potential. We hypothesize that, as in the brain, modulation of spontaneous oscillations may sustain learning in slime mould. This article is part of the theme issue 'Basal cognition: conceptual tools and the view from the single cell'.

Energetics of stochastic BCM type synaptic plasticity and storing of accurate information

Excitatory synaptic signaling in cortical circuits is thought to be metabolically expensive. Two fundamental brain functions, learning and memory, are associated with long-term synaptic plasticity, but we know very little about energetics of these slow biophysical processes. This study investigates the energy requirement of information storing in plastic synapses for an extended version of BCM plasticity with a decay term, stochastic noise, and nonlinear dependence of neuron’s firing rate on synaptic current (adaptation). It is shown that synaptic weights in this model exhibit bistability. In order to analyze the system analytically, it is reduced to a simple dynamic mean-field for a population averaged plastic synaptic current. Next, using the concepts of nonequilibrium thermodynamics, we derive the energy rate (entropy production rate) for plastic synapses and a corresponding Fisher information for coding presynaptic input. That energy, which is of chemical origin, is primarily used for battling fluctuations in the synaptic weights and presynaptic firing rates, and it increases steeply with synaptic weights, and more uniformly though nonlinearly with presynaptic firing. At the onset of synaptic bistability, Fisher information and memory lifetime both increase sharply, by a few orders of magnitude, but the plasticity energy rate changes only mildly. This implies that a huge gain in the precision of stored information does not have to cost large amounts of metabolic energy, which suggests that synaptic information is not directly limited by energy consumption. Interestingly, for very weak synaptic noise, such a limit on synaptic coding accuracy is imposed instead by a derivative of the plasticity energy rate with respect to the mean presynaptic firing, and this relationship has a general character that is independent of the plasticity type. An estimate for primate neocortex reveals that a relative metabolic cost of BCM type synaptic plasticity, as a fraction of neuronal cost related to fast synaptic transmission and spiking, can vary from negligible to substantial, depending on the synaptic noise level and presynaptic firing.

Effective thermodynamics of two interacting underdamped Brownian particles

Starting from the stochastic thermodynamics description of two coupled underdamped Brownian particles, we showcase and compare three different coarse-graining schemes leading to an effective thermodynamic description for the first of the two particles: marginalization over one particle, bipartite structure with information flows, and the Hamiltonian of mean force formalism. In the limit of time-scale separation where the second particle with a fast relaxation time scale locally equilibrates with respect to the coordinates of the first slowly relaxing particle, the effective thermodynamics resulting from the first and third approach are shown to capture the full thermodynamics and to coincide with each other. In the bipartite approach, the slow part does not, in general, allow for an exact thermodynamic description as the entropic exchange between the particles is ignored. Physically, the second particle effectively becomes part of the heat reservoir. In the limit where the second particle becomes heavy and thus deterministic, the effective thermodynamics of the first two coarse-graining methods coincide with the full one. The Hamiltonian of mean force formalism, however, is shown to be incompatible with that limit. Physically, the second particle becomes a work source. These theoretical results are illustrated using an exactly solvable harmonic model.

Prediction and Dissipation in Nonequilibrium Molecular Sensors: Conditionally Markovian Channels Driven by Memoryful Environments

Biological sensors must often predict their input while operating under metabolic constraints. However, determining whether or not a particular sensor is evolved or designed to be accurate and efficient is challenging. This arises partly from the functional constraints being at cross purposes and partly since quantifying the prediction performance of even in silico sensors can require prohibitively long simulations, especially when highly complex environments drive sensors out of equilibrium. To circumvent these difficulties, we develop new expressions for the prediction accuracy and thermodynamic costs of the broad class of conditionally Markovian sensors subject to complex, correlated (unifilar hidden semi-Markov) environmental inputs in nonequilibrium steady state. Predictive metrics include the instantaneous memory and the total predictable information (the mutual information between present sensor state and input future), while dissipation metrics include power extracted from the environment and the nonpredictive information rate. Success in deriving these formulae relies on identifying the environment's causal states, the input's minimal sufficient statistics for prediction. Using these formulae, we study large random channels and the simplest nontrivial biological sensor model-that of a Hill molecule, characterized by the number of ligands that bind simultaneously-the sensor's cooperativity. We find that the seemingly impoverished Hill molecule can capture an order of magnitude more predictable information than large random channels.

Dissipation in Non-Steady State Regulatory Circuits

In order to respond to environmental signals, cells often use small molecular circuits to transmit information about their surroundings. Recently, motivated by specific examples in signaling and gene regulation, a body of work has focused on the properties of circuits that function out of equilibrium and dissipate energy. We briefly review the probabilistic measures of information and dissipation and use simple models to discuss and illustrate trade-offs between information and dissipation in biological circuits. We find that circuits with non-steady state initial conditions can transmit more information at small readout delays than steady state circuits. The dissipative cost of this additional information proves marginal compared to the steady state dissipation. Feedback does not significantly increase the transmitted information for out of steady state circuits but does decrease dissipative costs. Lastly, we discuss the case of bursty gene regulatory circuits that, even in the fast switching limit, function out of equilibrium.

Uncertainty relations for time-delayed Langevin systems

The thermodynamic uncertainty relation, which establishes a universal trade-off between nonequilibrium current fluctuations and dissipation, has been found for various Markovian systems. However, this relation has not been revealed for non-Markovian systems; therefore, we investigate the thermodynamic uncertainty relation for time-delayed Langevin systems. We prove that the fluctuation of arbitrary dynamical observables is constrained by the Kullback-Leibler divergence between the distributions of the forward path and its reversed counterpart. Specifically, for observables that are antisymmetric under time reversal, the fluctuation is bounded from below by a function of a quantity that can be identified as a generalization of the total entropy production in Markovian systems. We also provide a lower bound for arbitrary observables that are odd under position reversal. The term in this bound reflects the extent to which the position symmetry has been broken in the system and can be positive even in equilibrium. Our results hold for finite observation times and a large class of time-delayed systems because detailed underlying dynamics are not required for the derivation. We numerically verify the derived uncertainty relations using two single time-delay systems and one distributed time-delay system.

Uncertainty relations in stochastic processes: An information inequality approach

The thermodynamic uncertainty relation is an inequality stating that it is impossible to attain higher precision than the bound defined by entropy production. In statistical inference theory, information inequalities assert that it is infeasible for any estimator to achieve an error smaller than the prescribed bound. Inspired by the similarity between the thermodynamic uncertainty relation and the information inequalities, we apply the latter to systems described by Langevin equations, and we derive the bound for the fluctuation of thermodynamic quantities. When applying the Cramér-Rao inequality, the obtained inequality reduces to the fluctuation-response inequality. We find that the thermodynamic uncertainty relation is a particular case of the Cramér-Rao inequality, in which the Fisher information is the total entropy production. Using the equality condition of the Cramér-Rao inequality, we find that the stochastic total entropy production is the only quantity that can attain equality in the thermodynamic uncertainty relation. Furthermore, we apply the Chapman-Robbins inequality and obtain a relation for the lower bound of the ratio between the variance and the sensitivity of systems in response to arbitrary perturbations.

Stochastic Thermodynamic Interpretation of Information Geometry

In recent years, the unified theory of information and thermodynamics has been intensively discussed in the context of stochastic thermodynamics. The unified theory reveals that information theory would be useful to understand nonstationary dynamics of systems far from equilibrium. In this Letter, we have found a new link between stochastic thermodynamics and information theory well-known as information geometry. By applying this link, an information geometric inequality can be interpreted as a thermodynamic uncertainty relationship between speed and thermodynamic cost. We have numerically applied an information geometric inequality to a thermodynamic model of a biochemical enzyme reaction.

Nonequilibrium thermodynamics of restricted Boltzmann machines

In this work, we analyze the nonequilibrium thermodynamics of a class of neural networks known as restricted Boltzmann machines (RBMs) in the context of unsupervised learning. We show how the network is described as a discrete Markov process and how the detailed balance condition and the Maxwell-Boltzmann equilibrium distribution are sufficient conditions for a complete thermodynamics description, including nonequilibrium fluctuation theorems. Numerical simulations in a fully trained RBM are performed and the heat exchange fluctuation theorem is verified with excellent agreement to the theory. We observe how the contrastive divergence functional, mostly used in unsupervised learning of RBMs, is closely related to nonequilibrium thermodynamic quantities. We also use the framework to interpret the estimation of the partition function of RBMs with the annealed importance sampling method from a thermodynamics standpoint. Finally, we argue that unsupervised learning of RBMs is equivalent to a work protocol in a system driven by the laws of thermodynamics in the absence of labeled data.