A Technical Critique of Some Parts of the Free Energy Principle

As One and Many: Relating Individual and Emergent Group-Level Generative Models in Active Inference

Active inference under the Free Energy Principle has been proposed as an across-scales compatible framework for understanding and modelling behaviour and self-maintenance. Crucially, a collective of active inference agents can, if they maintain a group-level Markov blanket, constitute a larger group-level active inference agent with a generative model of its own. This potential for computational scale-free structures speaks to the application of active inference to self-organizing systems across spatiotemporal scales, from cells to human collectives. Due to the difficulty of reconstructing the generative model that explains the behaviour of emergent group-level agents, there has been little research on this kind of multi-scale active inference. Here, we propose a data-driven methodology for characterising the relation between the generative model of a group-level agent and the dynamics of its constituent individual agents. We apply methods from computational cognitive modelling and computational psychiatry, applicable for active inference as well as other types of modelling approaches. Using a simple Multi-Armed Bandit task as an example, we employ the new ActiveInference.jl library for Julia to simulate a collective of agents who are equipped with a Markov blanket. We use sampling-based parameter estimation to make inferences about the generative model of the group-level agent, and we show that there is a non-trivial relationship between the generative models of individual agents and the group-level agent they constitute, even in this simple setting. Finally, we point to a number of ways in which this methodology might be applied to better understand the relations between nested active inference agents across scales.

Active Inference in Psychology and Psychiatry: Progress to Date?

The free energy principle is a formal theory of adaptive self-organising systems that emerged from statistical thermodynamics, machine learning and theoretical neuroscience and has since been translated into biologically plausible 'process theories' of cognition and behaviour, which fall under the banner of 'active inference'. Despite the promise this theory holds for theorising, research and practical applications in psychology and psychiatry, its impact on these disciplines has only now begun to bear fruit. The aim of this treatment is to consider the extent to which active inference has informed theoretical progress in psychology, before exploring its contributions to our understanding and treatment of psychopathology. Despite facing persistent translational obstacles, progress suggests that active inference has the potential to become a new paradigm that promises to unite psychology's subdisciplines, while readily incorporating the traditionally competing paradigms of evolutionary and developmental psychology. To date, however, progress towards this end has been slow. Meanwhile, the main outstanding question is whether this theory will make a positive difference through applications in clinical psychology, and its sister discipline of psychiatry.

Neuron-level Prediction and Noise can Implement Flexible Reward-Seeking Behavior

We show that neural networks can implement reward-seeking behavior using only local predictive updates and internal noise. These networks are capable of autonomous interaction with an environment and can switch between explore and exploit behavior, which we show is governed by attractor dynamics. Networks can adapt to changes in their architectures, environments, or motor interfaces without any external control signals. When networks have a choice between different tasks, they can form preferences that depend on patterns of noise and initialization, and we show that these preferences can be biased by network architectures or by changing learning rates. Our algorithm presents a flexible, biologically plausible way of interacting with environments without requiring an explicit environmental reward function, allowing for behavior that is both highly adaptable and autonomous. Code is available at https://github.com/ccli3896/PaN.

Hybrid Life: Integrating biological, artificial, and cognitive systems

Artificial life is a research field studying what processes and properties define life, based on a multidisciplinary approach spanning the physical, natural, and computational sciences. Artificial life aims to foster a comprehensive study of life beyond "life as we know it" and toward "life as it could be," with theoretical, synthetic, and empirical models of the fundamental properties of living systems. While still a relatively young field, artificial life has flourished as an environment for researchers with different backgrounds, welcoming ideas, and contributions from a wide range of subjects. Hybrid Life brings our attention to some of the most recent developments within the artificial life community, rooted in more traditional artificial life studies but looking at new challenges emerging from interactions with other fields. Hybrid Life aims to cover studies that can lead to an understanding, from first principles, of what systems are and how biological and artificial systems can interact and integrate to form new kinds of hybrid (living) systems, individuals, and societies. To do so, it focuses on three complementary perspectives: theories of systems and agents, hybrid augmentation, and hybrid interaction. Theories of systems and agents are used to define systems, how they differ (e.g., biological or artificial, autonomous, or nonautonomous), and how multiple systems relate in order to form new hybrid systems. Hybrid augmentation focuses on implementations of systems so tightly connected that they act as a single, integrated one. Hybrid interaction is centered around interactions within a heterogeneous group of distinct living and nonliving systems. After discussing some of the major sources of inspiration for these themes, we will focus on an overview of the works that appeared in Hybrid Life special sessions, hosted by the annual Artificial Life Conference between 2018 and 2022. This article is categorized under: Neuroscience > Cognition Philosophy > Artificial Intelligence Computer Science and Robotics > Robotics.

On Bayesian mechanics: a physics of and by beliefs

The aim of this paper is to introduce a field of study that has emerged over the last decade, called Bayesian mechanics. Bayesian mechanics is a probabilistic mechanics, comprising tools that enable us to model systems endowed with a particular partition (i.e. into particles), where the internal states (or the trajectories of internal states) of a particular system encode the parameters of beliefs about external states (or their trajectories). These tools allow us to write down mechanical theories for systems that look as if they are estimating posterior probability distributions over the causes of their sensory states. This provides a formal language for modelling the constraints, forces, potentials and other quantities determining the dynamics of such systems, especially as they entail dynamics on a space of beliefs (i.e. on a statistical manifold). Here, we will review the state of the art in the literature on the free energy principle, distinguishing between three ways in which Bayesian mechanics has been applied to particular systems (i.e. path-tracking, mode-tracking and mode-matching). We go on to examine a duality between the free energy principle and the constrained maximum entropy principle, both of which lie at the heart of Bayesian mechanics, and discuss its implications.

Free energy and inference in living systems

Organisms are non-equilibrium, stationary systems self-organized via spontaneous symmetry breaking and undergoing metabolic cycles with broken detailed balance in the environment. The thermodynamic free-energy (FE) principle describes an organism's homeostasis as the regulation of biochemical work constrained by the physical FE cost. By contrast, recent research in neuroscience and theoretical biology explains a higher organism's homeostasis and allostasis as Bayesian inference facilitated by the informational FE. As an integrated approach to living systems, this study presents an FE minimization theory overarching the essential features of both the thermodynamic and neuroscientific FE principles. Our results reveal that the perception and action of animals result from active inference entailed by FE minimization in the brain, and the brain operates as a Schrödinger's machine conducting the neural mechanics of minimizing sensory uncertainty. A parsimonious model suggests that the Bayesian brain develops the optimal trajectories in neural manifolds and induces a dynamic bifurcation between neural attractors in the process of active inference.

Regulative development as a model for origin of life and artificial life studies

Using the formal framework of the Free Energy Principle, we show how generic thermodynamic requirements on bidirectional information exchange between a system and its environment can generate complexity. This leads to the emergence of hierarchical computational architectures in systems that operate sufficiently far from thermal equilibrium. In this setting, the environment of any system increases its ability to predict system behavior by "engineering" the system towards increased morphological complexity and hence larger-scale, more macroscopic behaviors. When seen in this light, regulative development becomes an environmentally-driven process in which "parts" are assembled to produce a system with predictable behavior. We suggest on this basis that life is thermodynamically favorable and that, when designing artificial living systems, human engineers are acting like a generic "environment".

EEG Lempel-Ziv complexity varies with sleep stage, but does not seem to track dream experience

In a recent electroencephalography (EEG) sleep study inspired by complexity theories of consciousness, we found that multi-channel signal diversity progressively decreased from wakefulness to slow wave sleep, but failed to find any significant difference between dreaming and non-dreaming awakenings within the same sleep stage (NREM2). However, we did find that multi-channel Lempel-Ziv complexity (LZC) measured over the posterior cortex increased with more perceptual ratings of NREM2 dream experience along a thought-perceptual axis. In this follow-up study, we re-tested our previous findings, using a slightly different approach. Partial sleep-deprivation was followed by evening sleep experiments, with repeated awakenings and immediate dream reports. Participants reported whether they had been dreaming, and were asked to rate how diverse, vivid, perceptual, and thought-like the contents of their dreams were. High density (64 channel) EEG was recorded throughout the experiment, and mean single-channel LZC was calculated for each 30 s sleep epoch. LZC progressively decreased with depth of non-REM sleep. Surprisingly, estimated marginal mean LZC was slightly higher for NREM1 than for wakefulness, but the difference did not remain significant after adjusting for multiple comparisons. We found no significant difference in LZC between dream and non-dream awakenings, nor any significant relationship between LZC and subjective ratings of dream experience, within the same sleep stage (NREM2). The failure to reproduce our own previous finding of a positive correlation between posterior LZC and more perceptual dream experiences, or to find any other correlation between brain signal complexity and subjective experience within NREM2 sleep, raises the question of whether EEG LZC is really a reliable correlate of richness of experience as such, within the same sleep stage.

Recurrent, nonequilibrium systems and the Markov blanket assumption

Markov blankets - statistical independences between system and environment - have become popular to describe the boundaries of living systems under Bayesian views of cognition. The intuition behind Markov blankets originates from considering acyclic, atemporal networks. In contrast, living systems display recurrent, nonequilibrium interactions that generate pervasive couplings between system and environment, making Markov blankets highly unusual and restricted to particular cases.

The Emperor Is Naked: Replies to commentaries on the target article

The 35 commentaries cover a wide range of topics and take many different stances on the issues explored by the target article. We have organised our response to the commentaries around three central questions: Are Friston blankets just Pearl blankets? What ontological and metaphysical commitments are implied by the use of Friston blankets? What kind of explanatory work are Friston blankets capable of? We conclude our reply with a short critical reflection on the indiscriminate use of both Markov blankets and the free energy principle.

The emperor has no blanket!

While we applaud Bruineberg et al.'s analysis of the differences between Markov blankets and Friston blankets, we think it is not carried out to its ultimate consequences. There are reasons to think that, once Friston blankets are accepted as a theoretical construct, they do not do the work proponents of free energy principle (FEP) attribute to them. The emperor is indeed naked.

Embracing sensorimotor history: Time-synchronous and time-unrolled Markov blankets in the free-energy principle

The free-energy principle (FEP) builds on an assumption that sensor-motor loops exhibit Markov blankets in stationary state. We argue that there is rarely reason to assume a system's internal and external states are conditionally independent given the sensorimotor states, and often reason to assume otherwise. However, under mild assumptions internal and external states are conditionally independent given the sensorimotor history.

A continuity of Markov blanket interpretations under the free-energy principle

Bruineberg and colleagues helpfully distinguish between instrumental and ontological interpretations of Markov blankets, exposing the dangers of using the former to make claims about the latter. However, proposing a sharp distinction neglects the value of recognising a continuum spanning from instrumental to ontological. This value extends to the related distinction between "being" and "having" a model.

On the Inherent Instability of Biocognition: Toward New Probability Models and Statistical Tools

A central conundrum enshrouds biocognition: almost all such phenomena are inherently unstable and must be constantly controlled by external regulatory machinery to ensure proper function, in much the same sense that blood pressure and the ‘stream of consciousness’ require persistent delicate regulation for the survival of higher organisms. Here, we derive the Data Rate Theorem of control theory that characterizes such instability via the Rate Distortion Theorem of information theory for adiabatically stationary nonergodic systems. We then outline a novel approach to building new statistical tools for data analysis based on those theorems, focusing on groupoid symmetry-breaking phase transitions characterized by Fisher Zero analogs.

George Orwell, objectivity, and the reality behind illusions

Illusions are commonly defined as departures of our percepts from the veridical representation of objective, common-sense reality. However, it has been claimed recently that this definition lacks validity, for example, on the grounds that external reality cannot possibly be represented truly by our sensory systems, and indeed may even be a fiction. Here, I first demonstrate how novelist George Orwell warned that such denials of objective reality are dangerous mistakes, in that they can lead to the suppression and even the atrophy of independent thought and critical evaluation. Second, anti-realists assume their opponents hold a fully reductionist metaphysics, in which fundamental physics describes the only ground truth, thereby placing it beyond direct human sensory observation. In contrast, I point to a more recent and commonly used alternative, non-reductive metaphysics. This ascribes real existence to many levels of dynamic systems of information, emerging progressively from the subatomic to the biological, psychological, social, and ecological. Within such a worldview the notion of objective reality is valid, it comes in part within the range of our senses, and thus a definition of illusions as kinds of deviations from veridical perception becomes possible again.

Asymptotic burnout and homeostatic awakening: a possible solution to the Fermi paradox?

Previous studies show that city metrics having to do with growth, productivity and overall energy consumption scale superlinearly, attributing this to the social nature of cities. Superlinear scaling results in crises called ‘singularities’, where population and energy demand tend to infinity in a finite amount of time, which must be avoided by ever more frequent ‘resets’ or innovations that postpone the system's collapse. Here, we place the emergence of cities and planetary civilizations in the context of major evolutionary transitions. With this perspective, we hypothesize that once a planetary civilization transitions into a state that can be described as one virtually connected global city, it will face an ‘asymptotic burnout’, an ultimate crisis where the singularity-interval time scale becomes smaller than the time scale of innovation. If a civilization develops the capability to understand its own trajectory, it will have a window of time to affect a fundamental change to prioritize long-term homeostasis and well-being over unyielding growth—a consciously induced trajectory change or ‘homeostatic awakening’. We propose a new resolution to the Fermi paradox: civilizations either collapse from burnout or redirect themselves to prioritizing homeostasis, a state where cosmic expansion is no longer a goal, making them difficult to detect remotely.

Consciousness as the Temporal Propagation of Information

Our ability to understand the mind and its relation to the body is highly dependent on the way we define consciousness and the lens through which we study it. We argue that looking at conscious experience from an information-theory perspective can help obtain a unified and parsimonious account of the mind. Today's dominant models consider consciousness to be a specialized function of the brain characterized by a discrete neural event. Against this background, we consider subjective experience through information theory, presenting consciousness as the propagation of information from the past to the future. We examine through this perspective major characteristics of consciousness. We demonstrate that without any additional assumptions, temporal continuity in perception can explain the emergence of volition, subjectivity, higher order thoughts, and body boundaries. Finally, we discuss the broader implications for the mind-body question and the appeal of embodied cognition.

How particular is the physics of the free energy principle?

The free energy principle (FEP) states that any dynamical system can be interpreted as performing Bayesian inference upon its surrounding environment. Although, in theory, the FEP applies to a wide variety of systems, there has been almost no direct exploration or demonstration of the principle in concrete systems. In this work, we examine in depth the assumptions required to derive the FEP in the simplest possible set of systems - weakly-coupled non-equilibrium linear stochastic systems. Specifically, we explore (i) how general the requirements imposed on the statistical structure of a system are and (ii) how informative the FEP is about the behaviour of such systems. We discover that two requirements of the FEP - the Markov blanket condition (i.e. a statistical boundary precluding direct coupling between internal and external states) and stringent restrictions on its solenoidal flows (i.e. tendencies driving a system out of equilibrium) - are only valid for a very narrow space of parameters. Suitable systems require an absence of perception-action asymmetries that is highly unusual for living systems interacting with an environment. More importantly, we observe that a mathematically central step in the argument, connecting the behaviour of a system to variational inference, relies on an implicit equivalence between the dynamics of the average states of a system with the average of the dynamics of those states. This equivalence does not hold in general even for linear stochastic systems, since it requires an effective decoupling from the system's history of interactions. These observations are critical for evaluating the generality and applicability of the FEP and indicate the existence of significant problems of the theory in its current form. These issues make the FEP, as it stands, not straightforwardly applicable to the simple linear systems studied here and suggest that more development is needed before the theory could be applied to the kind of complex systems that describe living and cognitive processes.

The Free Energy Principle for Perception and Action: A Deep Learning Perspective

The free energy principle, and its corollary active inference, constitute a bio-inspired theory that assumes biological agents act to remain in a restricted set of preferred states of the world, i.e., they minimize their free energy. Under this principle, biological agents learn a generative model of the world and plan actions in the future that will maintain the agent in an homeostatic state that satisfies its preferences. This framework lends itself to being realized in silico, as it comprehends important aspects that make it computationally affordable, such as variational inference and amortized planning. In this work, we investigate the tool of deep learning to design and realize artificial agents based on active inference, presenting a deep-learning oriented presentation of the free energy principle, surveying works that are relevant in both machine learning and active inference areas, and discussing the design choices that are involved in the implementation process. This manuscript probes newer perspectives for the active inference framework, grounding its theoretical aspects into more pragmatic affairs, offering a practical guide to active inference newcomers and a starting point for deep learning practitioners that would like to investigate implementations of the free energy principle.

Bayesian mechanics for stationary processes

This paper develops a Bayesian mechanics for adaptive systems. Firstly, we model the interface between a system and its environment with a Markov blanket. This affords conditions under which states internal to the blanket encode information about external states. Second, we introduce dynamics and represent adaptive systems as Markov blankets at steady state. This allows us to identify a wide class of systems whose internal states appear to infer external states, consistent with variational inference in Bayesian statistics and theoretical neuroscience. Finally, we partition the blanket into sensory and active states. It follows that active states can be seen as performing active inference and well-known forms of stochastic control (such as PID control), which are prominent formulations of adaptive behaviour in theoretical biology and engineering.

Is Consciousness First in Virtual Reality?

The prevailing scientific paradigm is that matter is primary and everything, including consciousness can be derived from the laws governing matter. Although the scientific explanation of consciousness on these lines has not been realized, in this view it is only a matter of time before consciousness will be explained through neurobiological activity in the brain, and nothing else. There is an alternative view that holds that it is fundamentally impossible to explain how subjectivity can arise solely out of material processes-"the hard problem of consciousness"-and instead consciousness should be regarded in itself as a primary force in nature. This view attempts to derive, for example, the laws of physics from models of consciousness, instead of the other way around. While as scientists we can understand and have an intuition for the first paradigm, it is very difficult to understand what "consciousness is primary" might mean since it has no intuitive scientific grounding. Here we show that worlds experienced through virtual reality (VR) are such that consciousness is a first order phenomenon. We discuss the Interface Theory of Perception which claims that in physical reality perceptions are not veridical and that we do not see the "truth" but that perception is based on evolutionary payoffs. We show that this theory may provide an accurate description of perception and consciousness within VR, and we put forward an experimental study that could throw light on this. We conclude that VR does offer an experimental frame that provides intuition with respect to the idea that "consciousness is first" and what this might mean regarding the perceived world. However, we do not draw any conclusions about the veracity of this notion with respect to physical reality or question the emergence of consciousness from brain function.

Consciousness Beyond Neural Fields: Expanding the Possibilities of What Has Not Yet Happened

In the field theories in physics, any particular region of the presumed space-time continuum and all interactions between elementary objects therein can be objectively measured and/or accounted for mathematically. Since this does not apply to any of the field theories, or any other neural theory, of consciousness, their explanatory power is limited. As discussed in detail herein, the matter is complicated further by the facts than any scientifically operational definition of consciousness is inevitably partial, and that the phenomenon has no spatial dimensionality. Under the light of insights from research on meditation and expanded consciousness, chronic pain syndrome, healthy aging, and eudaimonic well-being, we may conceive consciousness as a source of potential energy that has no clearly defined spatial dimensionality, but can produce significant changes in others and in the world, observable in terms of changes in time. It is argued that consciousness may have evolved to enable the human species to generate such changes in order to cope with unprecedented and/or unpredictable adversity. Such coping could, ultimately, include the conscious planning of our own extinction when survival on the planet is no longer an acceptable option.

The Markov blanket trick: On the scope of the free energy principle and active inference

The free energy principle (FEP) has been presented as a unified brain theory, as a general principle for the self-organization of biological systems, and most recently as a principle for a theory of every thing. Additionally, active inference has been proposed as the process theory entailed by FEP that is able to model the full range of biological and cognitive events. In this paper, we challenge these two claims. We argue that FEP is not the general principle it is claimed to be, and that active inference is not the all-encompassing process theory it is purported to be either. The core aspects of our argumentation are that (i) FEP is just a way to generalize Bayesian inference to all domains by the use of a Markov blanket formalism, a generalization we call the Markov blanket trick; and that (ii) active inference presupposes successful perception and action instead of explaining them.

Stochastic Chaos and Markov Blankets

In this treatment of random dynamical systems, we consider the existence-and identification-of conditional independencies at nonequilibrium steady-state. These independencies underwrite a particular partition of states, in which internal states are statistically secluded from external states by blanket states. The existence of such partitions has interesting implications for the information geometry of internal states. In brief, this geometry can be read as a physics of sentience, where internal states look as if they are inferring external states. However, the existence of such partitions-and the functional form of the underlying densities-have yet to be established. Here, using the Lorenz system as the basis of stochastic chaos, we leverage the Helmholtz decomposition-and polynomial expansions-to parameterise the steady-state density in terms of surprisal or self-information. We then show how Markov blankets can be identified-using the accompanying Hessian-to characterise the coupling between internal and external states in terms of a generalised synchrony or synchronisation of chaos. We conclude by suggesting that this kind of synchronisation may provide a mathematical basis for an elemental form of (autonomous or active) sentience in biology.

Some Interesting Observations on the Free Energy Principle

Biehl et al. (2021) present some interesting observations on an early formulation of the free energy principle. We use these observations to scaffold a discussion of the technical arguments that underwrite the free energy principle. This discussion focuses on solenoidal coupling between various (subsets of) states in sparsely coupled systems that possess a Markov blanket-and the distinction between exact and approximate Bayesian inference, implied by the ensuing Bayesian mechanics.

The two kinds of free energy and the Bayesian revolution

The concept of free energy has its origins in 19th century thermodynamics, but has recently found its way into the behavioral and neural sciences, where it has been promoted for its wide applicability and has even been suggested as a fundamental principle of understanding intelligent behavior and brain function. We argue that there are essentially two different notions of free energy in current models of intelligent agency, that can both be considered as applications of Bayesian inference to the problem of action selection: one that appears when trading off accuracy and uncertainty based on a general maximum entropy principle, and one that formulates action selection in terms of minimizing an error measure that quantifies deviations of beliefs and policies from given reference models. The first approach provides a normative rule for action selection in the face of model uncertainty or when information processing capabilities are limited. The second approach directly aims to formulate the action selection problem as an inference problem in the context of Bayesian brain theories, also known as Active Inference in the literature. We elucidate the main ideas and discuss critical technical and conceptual issues revolving around these two notions of free energy that both claim to apply at all levels of decision-making, from the high-level deliberation of reasoning down to the low-level information processing of perception.