Thermodynamic Cost and Benefit of Memory

Disentangled representations for causal cognition

Complex adaptive agents consistently achieve their goals by solving problems that seem to require an understanding of causal information, information pertaining to the causal relationships that exist among elements of combined agent-environment systems. Causal cognition studies and describes the main characteristics of causal learning and reasoning in human and non-human animals, offering a conceptual framework to discuss cognitive performances based on the level of apparent causal understanding of a task. Despite the use of formal intervention-based models of causality, including causal Bayesian networks, psychological and behavioural research on causal cognition does not yet offer a computational account that operationalises how agents acquire a causal understanding of the world seemingly from scratch, i.e. without a-priori knowledge of relevant features of the environment. Research on causality in machine and reinforcement learning, especially involving disentanglement as a candidate process to build causal representations, represents on the other hand a concrete attempt at designing artificial agents that can learn about causality, shedding light on the inner workings of natural causal cognition. In this work, we connect these two areas of research to build a unifying framework for causal cognition that will offer a computational perspective on studies of animal cognition, and provide insights in the development of new algorithms for causal reinforcement learning in AI.

Is stochastic thermodynamics the key to understanding the energy costs of computation?

The relationship between the thermodynamic and computational properties of physical systems has been a major theoretical interest since at least the 19th century. It has also become of increasing practical importance over the last half-century as the energetic cost of digital devices has exploded. Importantly, real-world computers obey multiple physical constraints on how they work, which affects their thermodynamic properties. Moreover, many of these constraints apply to both naturally occurring computers, like brains or Eukaryotic cells, and digital systems. Most obviously, all such systems must finish their computation quickly, using as few degrees of freedom as possible. This means that they operate far from thermal equilibrium. Furthermore, many computers, both digital and biological, are modular, hierarchical systems with strong constraints on the connectivity among their subsystems. Yet another example is that to simplify their design, digital computers are required to be periodic processes governed by a global clock. None of these constraints were considered in 20th-century analyses of the thermodynamics of computation. The new field of stochastic thermodynamics provides formal tools for analyzing systems subject to all of these constraints. We argue here that these tools may help us understand at a far deeper level just how the fundamental thermodynamic properties of physical systems are related to the computation they perform.

Information Thermodynamics: From Physics to Neuroscience

This paper provides a perspective on applying the concepts of information thermodynamics, developed recently in non-equilibrium statistical physics, to problems in theoretical neuroscience. Historically, information and energy in neuroscience have been treated separately, in contrast to physics approaches, where the relationship of entropy production with heat is a central idea. It is argued here that also in neural systems, information and energy can be considered within the same theoretical framework. Starting from basic ideas of thermodynamics and information theory on a classic Brownian particle, it is shown how noisy neural networks can infer its probabilistic motion. The decoding of the particle motion by neurons is performed with some accuracy, and it has some energy cost, and both can be determined using information thermodynamics. In a similar fashion, we also discuss how neural networks in the brain can learn the particle velocity and maintain that information in the weights of plastic synapses from a physical point of view. Generally, it is shown how the framework of stochastic and information thermodynamics can be used practically to study neural inference, learning, and information storing.

Efficiency of an autonomous, dynamic information engine operating on a single active particle

The Szilard engine stands as a compelling illustration of the intricate interplay between information and thermodynamics. While at thermodynamic equilibrium, the apparent breach of the second law of thermodynamics was reconciled by Landauer and Bennett's insights into memory writing and erasure, recent extensions of these concepts into regimes featuring active fluctuations have unveiled the prospect of exceeding Landauer's bound, capitalizing on information to divert free energy from dissipation towards useful work. To explore this question further, we investigate an autonomous dynamic information engine, addressing the thermodynamic consistency of work extraction and measurement costs by extending the phase space to incorporate an auxiliary system, which plays the role of an explicit measurement device. The nonreciprocal coupling between active particle and measurement device introduces a feedback control loop, and the cost of measurement is quantified through a suitably defined auxiliary entropy production. The study considers different measurement scenarios, highlighting the role of measurement precision in determining engine efficiency.

Unlocking the potential of information flow: Maximizing free-energy transduction in a model of an autonomous rotary molecular motor

Molecular motors fulfill critical functions within all living beings. Understanding their underlying working principles is therefore of great interest. Here we develop a simple model inspired by the two-component biomolecular motor F_{o}-F_{1} ATP synthase. We analyze its energetics and characterize information flows between the machine's components. At maximum output power we find that information transduction plays a minor role for free-energy transduction. However, when the two components are coupled to different environments (e.g., when in contact with heat baths at different temperatures), we show that information flow becomes a resource worth exploiting to maximize free-energy transduction. Our findings suggest that real-world powerful and efficient information engines could be found in machines whose components are subjected to fluctuations of different strength, since in this situation the benefit gained from using information for work extraction can outweigh the costs of information generation.

Information Engine in a Nonequilibrium Bath

Information engines can convert thermal fluctuations of a bath at temperature T into work at rates of order k_{B}T per relaxation time of the system. We show experimentally that such engines, when in contact with a bath that is out of equilibrium, can extract much more work. We place a heavy, micron-scale bead in a harmonic potential that ratchets up to capture favorable fluctuations. Adding a fluctuating electric field increases work extraction up to ten times, limited only by the strength of the applied field. Our results connect Maxwell's demon with energy harvesting and demonstrate that information engines in nonequilibrium baths can greatly outperform conventional engines.

Decomposing the Local Arrow of Time in Interacting Systems

We show that the evidence for a local arrow of time, which is equivalent to the entropy production in thermodynamic systems, can be decomposed. In a system with many degrees of freedom, there is a term that arises from the irreversible dynamics of the individual variables, and then a series of non-negative terms contributed by correlations among pairs, triplets, and higher-order combinations of variables. We illustrate this decomposition on simple models of noisy logical computations, and then apply it to the analysis of patterns of neural activity in the retina as it responds to complex dynamic visual scenes. We find that neural activity breaks detailed balance even when the visual inputs do not, and that this irreversibility arises primarily from interactions between pairs of neurons.

Emergence of local irreversibility in complex interacting systems

Living systems are fundamentally irreversible, breaking detailed balance and establishing an arrow of time. But how does the evident arrow of time for a whole system arise from the interactions among its multiple elements? We show that the local evidence for the arrow of time, which is the entropy production for thermodynamic systems, can be decomposed. First, it can be split into two components: an independent term reflecting the dynamics of individual elements and an interaction term driven by the dependencies among elements. Adapting tools from nonequilibrium physics, we further decompose the interaction term into contributions from pairs of elements, triplets, and higher-order terms. We illustrate our methods on models of cellular sensing and logical computations, as well as on patterns of neural activity in the retina as it responds to visual inputs. We find that neural activity can define the arrow of time even when the visual inputs do not, and that the dominant contribution to this breaking of detailed balance comes from interactions among pairs of neurons.

Pareto-Optimal Clustering with the Primal Deterministic Information Bottleneck

At the heart of both lossy compression and clustering is a trade-off between the fidelity and size of the learned representation. Our goal is to map out and study the Pareto frontier that quantifies this trade-off. We focus on the optimization of the Deterministic Information Bottleneck (DIB) objective over the space of hard clusterings. To this end, we introduce the primal DIB problem, which we show results in a much richer frontier than its previously studied Lagrangian relaxation when optimized over discrete search spaces. We present an algorithm for mapping out the Pareto frontier of the primal DIB trade-off that is also applicable to other two-objective clustering problems. We study general properties of the Pareto frontier, and we give both analytic and numerical evidence for logarithmic sparsity of the frontier in general. We provide evidence that our algorithm has polynomial scaling despite the super-exponential search space, and additionally, we propose a modification to the algorithm that can be used where sampling noise is expected to be significant. Finally, we use our algorithm to map the DIB frontier of three different tasks: compressing the English alphabet, extracting informative color classes from natural images, and compressing a group theory-inspired dataset, revealing interesting features of frontier, and demonstrating how the structure of the frontier can be used for model selection with a focus on points previously hidden by the cloak of the convex hull.

Bayesian mechanics for stationary processes

This paper develops a Bayesian mechanics for adaptive systems. Firstly, we model the interface between a system and its environment with a Markov blanket. This affords conditions under which states internal to the blanket encode information about external states. Second, we introduce dynamics and represent adaptive systems as Markov blankets at steady state. This allows us to identify a wide class of systems whose internal states appear to infer external states, consistent with variational inference in Bayesian statistics and theoretical neuroscience. Finally, we partition the blanket into sensory and active states. It follows that active states can be seen as performing active inference and well-known forms of stochastic control (such as PID control), which are prominent formulations of adaptive behaviour in theoretical biology and engineering.

Geometric Brownian information engine: Upper bound of the achievable work under feedback control

We design a geometric Brownian information engine by considering overdamped Brownian particles inside a two-dimensional monolobal confinement with irregular width along the transport direction. Under such detention, particles experience an effective entropic potential which has a logarithmic form. We employ a feedback control protocol as an outcome of error-free position measurement. The protocol comprises three stages: measurement, feedback, and relaxation. We reposition the center of the confinement to the measurement distance (x_p) instantaneously when the position of the trapped particle crosses x_p for the first time. Then, the particle is allowed for thermal relaxation. We calculate the extractable work, total information, and unavailable information associated with the feedback control using this equilibrium probability distribution function. We find the exact analytical value of the upper bound of extractable work as (53-2ln2)k_BT. We introduce a constant force G downward to the transverse coordinate (y). A change in G alters the effective potential of the system and tunes the relative dominance of entropic and energetic contributions in it. The upper bound of the achievable work shows a crossover from (53-2ln2)k_BT to 12k_BT when the system changes from an entropy-dominated regime to an energy-dominated one. Compared to an energetic analog, the loss of information during the relaxation process is higher in the entropy-dominated region, which accredits the less value in achievable work. Theoretical predictions are in good agreement with the Langevin dynamics simulation studies.

Optimal work extraction and mutual information in a generalized Szilárd engine

A 1929 Gedankenexperiment proposed by Szilárd, often referred to as "Szilárd's engine", has served as a foundation for computing fundamental thermodynamic bounds to information processing. While Szilárd's original box could be partitioned into two halves and contains one gas molecule, we calculate here the maximal average work that can be extracted in a system with N particles and q partitions, given an observer which counts the molecules in each partition, and given a work extraction mechanism that is limited to pressure equalization. We find that the average extracted work is proportional to the mutual information between the one-particle position and the vector containing the counts of how many particles are in each partition. We optimize this quantity over the initial locations of the dividing walls, and find that there exists a critical number of particles N^{★}(q) below which the extracted work is maximized by a symmetric configuration of the q partitions, and above which the optimal partitioning is asymmetric. Overall, the average extracted work is maximized for a number of particles N[over ̂](q)<N^{★}(q), with a symmetric partition. We calculate asymptotic values for N→∞.

Maximizing power and velocity of an information engine

Information-driven engines that rectify thermal fluctuations are a modern realization of the Maxwell-demon thought experiment. We introduce a simple design based on a heavy colloidal particle, held by an optical trap and immersed in water. Using a carefully designed feedback loop, our experimental realization of an "information ratchet" takes advantage of favorable "up" fluctuations to lift a weight against gravity, storing potential energy without doing external work. By optimizing the ratchet design for performance via a simple theory, we find that the rate of work storage and velocity of directed motion are limited only by the physical parameters of the engine: the size of the particle, stiffness of the ratchet spring, friction produced by the motion, and temperature of the surrounding medium. Notably, because performance saturates with increasing frequency of observations, the measurement process is not a limiting factor. The extracted power and velocity are at least an order of magnitude higher than in previously reported engines.

Variations on a demonic theme: Szilard's other engines

Szilard's now-famous single-molecule engine was only the first of three constructions he introduced in 1929 to resolve several challenges arising from Maxwell's demon paradox. Given that it has been thoroughly analyzed, we analyze Szilard's remaining two demon models. We show that the second one, though a markedly different implementation employing a population of distinct molecular species and semipermeable membranes, is informationally and thermodynamically equivalent to an ideal gas of the single-molecule engines. One concludes that (i) it reduces to a chaotic dynamical system-called the Szilard Map, a composite of three piecewise linear maps and associated thermodynamic transformations that implement measurement, control, and erasure; (ii) its transitory functioning as an engine that converts disorganized heat energy to work is governed by the Kolmogorov-Sinai entropy rate; (iii) the demon's minimum necessary "intelligence" for optimal functioning is given by the engine's statistical complexity; and (iv) its functioning saturates thermodynamic bounds and so it is a minimal, optimal implementation. We show that Szilard's third construction is rather different and addresses the fundamental issue raised by the first two: the link between entropy production and the measurement task required to implement either of his engines. The analysis gives insight into designing and implementing novel nanoscale information engines by investigating the relationships between the demon's memory, the nature of the "working fluid," and the thermodynamic costs of erasure and measurement.