Prefrontal Computation as Active Inference

Developing algorithmic psychiatry via multi-level spanning computational models

Modern psychiatry faces challenges in translating neurobiological insights into treatments for severe illnesses. The mid-20th century witnessed the rise of molecular mechanisms as pathophysiological and treatment models, with recent holistic proposals keeping this focus unaltered. In this perspective, we explore how psychiatry can utilize systems neuroscience to develop a vertically integrated understanding of brain function to inform treatment. Using schizophrenia as a case study, we discuss scale-related challenges faced by researchers studying molecules, circuits, networks, and cognition and clinicians operating within existing frameworks. We emphasize computation as a bridging language, with algorithmic models like hierarchical predictive processing offering explanatory potential for targeted interventions. Developing such models will not only facilitate new interventions but also optimize combining existing treatments by predicting their multi-level effects. We conclude with the prognosis that the future is bright, but that continued investment in research closely driven by clinical realities will be critical.

Dynamic planning in hierarchical active inference

By dynamic planning, we refer to the ability of the human brain to infer and impose motor trajectories related to cognitive decisions. A recent paradigm, active inference, brings fundamental insights into the adaptation of biological organisms, constantly striving to minimize prediction errors to restrict themselves to life-compatible states. Over the past years, many studies have shown how human and animal behaviors could be explained in terms of active inference - either as discrete decision-making or continuous motor control - inspiring innovative solutions in robotics and artificial intelligence. Still, the literature lacks a comprehensive outlook on effectively planning realistic actions in changing environments. Setting ourselves the goal of modeling complex tasks such as tool use, we delve into the topic of dynamic planning in active inference, keeping in mind two crucial aspects of biological behavior: the capacity to understand and exploit affordances for object manipulation, and to learn the hierarchical interactions between the self and the environment, including other agents. We start from a simple unit and gradually describe more advanced structures, comparing recently proposed design choices and providing basic examples. This study distances itself from traditional views centered on neural networks and reinforcement learning, and points toward a yet unexplored direction in active inference: hybrid representations in hierarchical models.

RDoC Framework Through the Lens of Predictive Processing: Focusing on Cognitive Systems Domain

In response to shortcomings of the current classification system in translating discoveries from basic science to clinical applications, NIMH offers a new framework for studying mental health disorders called Research Domain Criteria (RDoC). This framework holds a multidimensional outlook on psychopathologies focusing on functional domains of behavior and their implementing neural circuits. In parallel, the Predictive Processing (PP) framework stands as a leading theory of human brain function, offering a unified explanation for various types of information processing in the brain. While both frameworks share an interest in studying psychopathologies based on pathophysiology, their integration still needs to be explored. Here, we argued in favor of the explanatory power of PP to be a groundwork for the RDoC matrix in validating its constructs and creating testable hypotheses about mechanistic interactions between molecular biomarkers and clinical traits. Together, predictive processing may serve as a foundation for achieving the goals of the RDoC framework.

Slow but flexible or fast but rigid? Discrete and continuous processes compared

A tradeoff exists when dealing with complex tasks composed of multiple steps. High-level cognitive processes can find the best sequence of actions to achieve a goal in uncertain environments, but they are slow and require significant computational demand. In contrast, lower-level processing allows reacting to environmental stimuli rapidly, but with limited capacity to determine optimal actions or to replan when expectations are not met. Through reiteration of the same task, biological organisms find the optimal tradeoff: from action primitives, composite trajectories gradually emerge by creating task-specific neural structures. The two frameworks of active inference - a recent brain paradigm that views action and perception as subject to the same free energy minimization imperative - well capture high-level and low-level processes of human behavior, but how task specialization occurs in these terms is still unclear. In this study, we compare two strategies on a dynamic pick-and-place task: a hybrid (discrete-continuous) model with planning capabilities and a continuous-only model with fixed transitions. Both models rely on a hierarchical (intrinsic and extrinsic) structure, well suited for defining reaching and grasping movements, respectively. Our results show that continuous-only models perform better and with minimal resource expenditure but at the cost of less flexibility. Finally, we propose how discrete actions might lead to continuous attractors and compare the two frameworks with different motor learning phases, laying the foundations for further studies on bio-inspired task adaptation.

Stress-induced failure of embodied cognition: A general model

We derive the classic, ubiquitous, but enigmatic Yerkes-Dodson effect of applied stress on real-world performance in a highly natural manner from fundamental assumptions on cognition and its dynamics, as constrained by the asymptotic limit theorems of information and control theories. We greatly extend the basic approach by showing how differences in an underlying probability model can affect the dynamics of decision across a broad range of cognitive enterprise. Most particularly, however, this development may help inform our understanding of the different expressions of human psychopathology. A 'thin tailed' underlying distribution appears to characterize expression of 'ordinary' situational depression/anxiety symptoms of conditions like burnout induced by toxic stress. A 'fat tailed' underlying distribution appears to be associated with brain structure and function abnormalities leading to serious mental illness and poor decision making where symptoms are not only emerging in the setting of severe stress but may also appear in a highly punctuated manner at relatively lower levels of stress. A simple hierarchical optimization shows how environmental 'shadow price' constraints can buffer or aggravate the effects of stress and arousal. Extension of the underlying theory to other patterns of pathology, like immune disorders and premature aging, seems apt. Applications to the punctuated dynamics of institutional cognition under stress also appear possible. Ultimately, the probability models studied here can be converted to new statistical tools for the analysis of observational and experimental data.

An Introduction to Predictive Processing Models of Perception and Decision-Making

The predictive processing framework includes a broad set of ideas, which might be articulated and developed in a variety of ways, concerning how the brain may leverage predictive models when implementing perception, cognition, decision-making, and motor control. This article provides an up-to-date introduction to the two most influential theories within this framework: predictive coding and active inference. The first half of the paper (Sections 2-5) reviews the evolution of predictive coding, from early ideas about efficient coding in the visual system to a more general model encompassing perception, cognition, and motor control. The theory is characterized in terms of the claims it makes at Marr's computational, algorithmic, and implementation levels of description, and the conceptual and mathematical connections between predictive coding, Bayesian inference, and variational free energy (a quantity jointly evaluating model accuracy and complexity) are explored. The second half of the paper (Sections 6-8) turns to recent theories of active inference. Like predictive coding, active inference models assume that perceptual and learning processes minimize variational free energy as a means of approximating Bayesian inference in a biologically plausible manner. However, these models focus primarily on planning and decision-making processes that predictive coding models were not developed to address. Under active inference, an agent evaluates potential plans (action sequences) based on their expected free energy (a quantity that combines anticipated reward and information gain). The agent is assumed to represent the world as a partially observable Markov decision process with discrete time and discrete states. Current research applications of active inference models are described, including a range of simulation work, as well as studies fitting models to empirical data. The paper concludes by considering future research directions that will be important for further development of both models.

Relative fluency (unfelt vs felt) in active inference

For a growing number of researchers, it is now accepted that the brain is a predictive organ that predicts the content of the sensorium and crucially the precision of-or confidence in-its own predictions. In order to predict the precision of its predictions, the brain has to infer the reliability of its own beliefs. This means that our brains have to recognise the precision of their predictions or, at least, their accuracy. In this paper, we argue that fluency is product of this recognition process. In short, to recognise fluency is to infer that we have a precise 'grip' on the unfolding processes that generate our sensations. More specifically, we propose that it is changes in fluency - from unfelt to felt - that are both recognised and realised when updating predictions about precision. Unfelt fluency orients attention to unpredicted sensations, while felt fluency supervenes on-and contextualises-unfelt fluency; thereby rendering certain attentional processes, phenomenologically opaque. As such, fluency underwrites the precision we place in our predictions and therefore acts upon our perceptual inferences. Hence, the causes of conscious subjective inference have unconscious perceptual precursors.

Functional connectivity abnormalities of brain networks in obsessive–compulsive disorder: a systematic review

Obsessive-compulsive disorder (OCD) is characterized by cognitive abnormalities encompassing several executive processes. Neuroimaging studies highlight functional abnormalities of executive fronto-parietal network (FPN) and default-mode network (DMN) in OCD patients, as well as of the prefrontal cortex (PFC) more specifically. We aim at assessing the presence of functional connectivity (FC) abnormalities of intrinsic brain networks and PFC in OCD, possibly underlying specific computational impairments and clinical manifestations. A systematic review of resting-state fMRI studies investigating FC was conducted in unmedicated OCD patients by querying three scientific databases (PubMed, Scopus, PsycInfo) up to July 2022 (search terms: “obsessive–compulsive disorder” AND “resting state” AND “fMRI” AND “function* *connect*” AND “task-positive” OR “executive” OR “central executive” OR “executive control” OR “executive-control” OR “cognitive control” OR “attenti*” OR “dorsal attention” OR “ventral attention” OR “frontoparietal” OR “fronto-parietal” OR “default mode” AND “network*” OR “system*”). Collectively, 20 studies were included. A predominantly reduced FC of DMN – often related to increased symptom severity – emerged. Additionally, intra-network FC of FPN was predominantly increased and often positively related to clinical scores. Concerning PFC, a predominant hyper-connectivity of right-sided prefrontal links emerged. Finally, FC of lateral prefrontal areas correlated with specific symptom dimensions. Several sources of heterogeneity in methodology might have affected results in unpredictable ways and were discussed. Such findings might represent endophenotypes of OCD manifestations, possibly reflecting computational impairments and difficulties in engaging in self-referential processes or in disengaging from cognitive control and monitoring processes.

Flexible intentions: An Active Inference theory

We present a normative computational theory of how the brain may support visually-guided goal-directed actions in dynamically changing environments. It extends the Active Inference theory of cortical processing according to which the brain maintains beliefs over the environmental state, and motor control signals try to fulfill the corresponding sensory predictions. We propose that the neural circuitry in the Posterior Parietal Cortex (PPC) compute flexible intentions-or motor plans from a belief over targets-to dynamically generate goal-directed actions, and we develop a computational formalization of this process. A proof-of-concept agent embodying visual and proprioceptive sensors and an actuated upper limb was tested on target-reaching tasks. The agent behaved correctly under various conditions, including static and dynamic targets, different sensory feedbacks, sensory precisions, intention gains, and movement policies; limit conditions were individuated, too. Active Inference driven by dynamic and flexible intentions can thus support goal-directed behavior in constantly changing environments, and the PPC might putatively host its core intention mechanism. More broadly, the study provides a normative computational basis for research on goal-directed behavior in end-to-end settings and further advances mechanistic theories of active biological systems.

Integrated world modeling theory expanded: Implications for the future of consciousness

Integrated world modeling theory (IWMT) is a synthetic theory of consciousness that uses the free energy principle and active inference (FEP-AI) framework to combine insights from integrated information theory (IIT) and global neuronal workspace theory (GNWT). Here, I first review philosophical principles and neural systems contributing to IWMT's integrative perspective. I then go on to describe predictive processing models of brains and their connections to machine learning architectures, with particular emphasis on autoencoders (perceptual and active inference), turbo-codes (establishment of shared latent spaces for multi-modal integration and inferential synergy), and graph neural networks (spatial and somatic modeling and control). Future directions for IIT and GNWT are considered by exploring ways in which modules and workspaces may be evaluated as both complexes of integrated information and arenas for iterated Bayesian model selection. Based on these considerations, I suggest novel ways in which integrated information might be estimated using concepts from probabilistic graphical models, flow networks, and game theory. Mechanistic and computational principles are also considered with respect to the ongoing debate between IIT and GNWT regarding the physical substrates of different kinds of conscious and unconscious phenomena. I further explore how these ideas might relate to the "Bayesian blur problem," or how it is that a seemingly discrete experience can be generated from probabilistic modeling, with some consideration of analogies from quantum mechanics as potentially revealing different varieties of inferential dynamics. I go on to describe potential means of addressing critiques of causal structure theories based on network unfolding, and the seeming absurdity of conscious expander graphs (without cybernetic symbol grounding). Finally, I discuss future directions for work centered on attentional selection and the evolutionary origins of consciousness as facilitated "unlimited associative learning." While not quite solving the Hard problem, this article expands on IWMT as a unifying model of consciousness and the potential future evolution of minds.

Geometry of neural computation unifies working memory and planning

Real-world tasks require coordination of working memory, decision-making, and planning, yet these cognitive functions have disproportionately been studied as independent modular processes in the brain. Here, we propose that contingency representations, defined as mappings for how future behaviors depend on upcoming events, can unify working memory and planning computations. We designed a task capable of disambiguating distinct types of representations. In task-optimized recurrent neural networks, we investigated possible circuit mechanisms for contingency representations and found that these representations can explain neurophysiological observations from the prefrontal cortex during working memory tasks. Our experiments revealed that human behavior is consistent with contingency representations and not with traditional sensory models of working memory. Finally, we generated falsifiable predictions for neural data to identify contingency representations in neural data and to dissociate different models of working memory. Our findings characterize a neural representational strategy that can unify working memory, planning, and context-dependent decision-making.

An active inference model of conscious access: How cognitive action selection reconciles the results of report and no-report paradigms

Cognitive theories of consciousness, such as global workspace theory and higher-order theories, posit that frontoparietal circuits play a crucial role in conscious access. However, recent studies using no-report paradigms have posed a challenge to cognitive theories by demonstrating conscious accessibility in the apparent absence of prefrontal cortex (PFC) activation. To address this challenge, this paper presents a computational model of conscious access, based upon active inference, that treats working memory gating as a cognitive action. We simulate a visual masking task and show that late P3b-like event-related potentials (ERPs), and increased PFC activity, are induced by the working memory demands of self-report generation. When reporting demands are removed, these late ERPs vanish and PFC activity is reduced. These results therefore reproduce, and potentially explain, results from no-report paradigms. However, even without reporting demands, our model shows that simulated PFC activity on visible stimulus trials still crosses the threshold for reportability - maintaining the link between PFC and conscious access. Therefore, our simulations show that evidence provided by no-report paradigms does not necessarily contradict cognitive theories of consciousness.

World model learning and inference

Understanding information processing in the brain-and creating general-purpose artificial intelligence-are long-standing aspirations of scientists and engineers worldwide. The distinctive features of human intelligence are high-level cognition and control in various interactions with the world including the self, which are not defined in advance and are vary over time. The challenge of building human-like intelligent machines, as well as progress in brain science and behavioural analyses, robotics, and their associated theoretical formalisations, speaks to the importance of the world-model learning and inference. In this article, after briefly surveying the history and challenges of internal model learning and probabilistic learning, we introduce the free energy principle, which provides a useful framework within which to consider neuronal computation and probabilistic world models. Next, we showcase examples of human behaviour and cognition explained under that principle. We then describe symbol emergence in the context of probabilistic modelling, as a topic at the frontiers of cognitive robotics. Lastly, we review recent progress in creating human-like intelligence by using novel probabilistic programming languages. The striking consensus that emerges from these studies is that probabilistic descriptions of learning and inference are powerful and effective ways to create human-like artificial intelligent machines and to understand intelligence in the context of how humans interact with their world.

Towards a computational phenomenology of mental action: modelling meta-awareness and attentional control with deep parametric active inference

Meta-awareness refers to the capacity to explicitly notice the current content of consciousness and has been identified as a key component for the successful control of cognitive states, such as the deliberate direction of attention. This paper proposes a formal model of meta-awareness and attentional control using hierarchical active inference. To do so, we cast mental action as policy selection over higher-level cognitive states and add a further hierarchical level to model meta-awareness states that modulate the expected confidence (precision) in the mapping between observations and hidden cognitive states. We simulate the example of mind-wandering and its regulation during a task involving sustained selective attention on a perceptual object. This provides a computational case study for an inferential architecture that is apt to enable the emergence of these central components of human phenomenology, namely, the ability to access and control cognitive states. We propose that this approach can be generalized to other cognitive states, and hence, this paper provides the first steps towards the development of a computational phenomenology of mental action and more broadly of our ability to monitor and control our own cognitive states. Future steps of this work will focus on fitting the model with qualitative, behavioural, and neural data.

Memory and Markov Blankets

In theoretical biology, we are often interested in random dynamical systems-like the brain-that appear to model their environments. This can be formalized by appealing to the existence of a (possibly non-equilibrium) steady state, whose density preserves a conditional independence between a biological entity and its surroundings. From this perspective, the conditioning set, or Markov blanket, induces a form of vicarious synchrony between creature and world-as if one were modelling the other. However, this results in an apparent paradox. If all conditional dependencies between a system and its surroundings depend upon the blanket, how do we account for the mnemonic capacity of living systems? It might appear that any shared dependence upon past blanket states violates the independence condition, as the variables on either side of the blanket now share information not available from the current blanket state. This paper aims to resolve this paradox, and to demonstrate that conditional independence does not preclude memory. Our argument rests upon drawing a distinction between the dependencies implied by a steady state density, and the density dynamics of the system conditioned upon its configuration at a previous time. The interesting question then becomes: What determines the length of time required for a stochastic system to 'forget' its initial conditions? We explore this question for an example system, whose steady state density possesses a Markov blanket, through simple numerical analyses. We conclude with a discussion of the relevance for memory in cognitive systems like us.

A Predictive Processing Account of Card Sorting: Fast Proactive and Reactive Frontoparietal Cortical Dynamics during Inference and Learning of Perceptual Categories

For decades, a common assumption in cognitive neuroscience has been that prefrontal executive control is mainly engaged during target detection [Posner, M. I., & Petersen, S. E. The attention system of the human brain. Annual Review of Neuroscience, 13, 25-42, 1990]. More recently, predictive processing theories of frontal function under the Bayesian brain hypothesis emphasize a key role of proactive control for anticipatory action selection (i.e., planning as active inference). Here, we review evidence of fast and widespread EEG and magnetoencephalographic fronto-temporo-parietal cortical activations elicited by feedback cues and target cards in the Wisconsin Card Sorting Test. This evidence is best interpreted when considering negative and positive feedback as predictive cues (i.e., sensory outcomes) for proactively updating beliefs about unknown perceptual categories. Such predictive cues inform posterior beliefs about high-level hidden categories governing subsequent response selection at target onset. Quite remarkably, these new views concur with Don Stuss' early findings concerning two broad classes of P300 cortical responses evoked by feedback cues and target cards in a computerized Wisconsin Card Sorting Test analogue. Stuss' discussion of those P300 responses-in terms of the resolution of uncertainty about response (policy) selection as well as the participants' expectancies for future perceptual or motor activities and their timing-was prescient of current predictive processing and active (Bayesian) inference theories. From these new premises, a domain-general frontoparietal cortical network is rapidly engaged during two temporarily distinct stages of inference and learning of perceptual categories that underwrite goal-directed card sorting behavior, and they each engage prefrontal executive functions in fundamentally distinct ways.

Evidence and implications of abnormal predictive coding in dementia

The diversity of cognitive deficits and neuropathological processes associated with dementias has encouraged divergence in pathophysiological explanations of disease. Here, we review an alternative framework that emphasizes convergent critical features of cognitive pathophysiology. Rather than the loss of ‘memory centres’ or ‘language centres’, or singular neurotransmitter systems, cognitive deficits are interpreted in terms of aberrant predictive coding in hierarchical neural networks. This builds on advances in normative accounts of brain function, specifically the Bayesian integration of beliefs and sensory evidence in which hierarchical predictions and prediction errors underlie memory, perception, speech and behaviour. We describe how analogous impairments in predictive coding in parallel neurocognitive systems can generate diverse clinical phenomena, including the characteristics of dementias. The review presents evidence from behavioural and neurophysiological studies of perception, language, memory and decision-making. The reformulation of cognitive deficits in terms of predictive coding has several advantages. It brings diverse clinical phenomena into a common framework; it aligns cognitive and movement disorders; and it makes specific predictions on cognitive physiology that support translational and experimental medicine studies. The insights into complex human cognitive disorders from the predictive coding framework may therefore also inform future therapeutic strategies.

Simulating the computational mechanisms of cognitive and behavioral psychotherapeutic interventions: insights from active inference

Cognitive-behavioral therapy (CBT) leverages interactions between thoughts, feelings, and behaviors. To deepen understanding of these interactions, we present a computational (active inference) model of CBT that allows formal simulations of interactions between cognitive interventions (i.e., cognitive restructuring) and behavioral interventions (i.e., exposure) in producing adaptive behavior change (i.e., reducing maladaptive avoidance behavior). Using spider phobia as a concrete example of maladaptive avoidance more generally, we show simulations indicating that when conscious beliefs about safety/danger have strong interactions with affective/behavioral outcomes, behavioral change during exposure therapy is mediated by changes in these beliefs, preventing generalization. In contrast, when these interactions are weakened, and cognitive restructuring only induces belief uncertainty (as opposed to strong safety beliefs), behavior change leads to generalized learning (i.e., "over-writing" the implicit beliefs about action-outcome mappings that directly produce avoidance). The individual is therefore equipped to face any new context, safe or dangerous, remaining in a content state without the need for avoidance behavior-increasing resilience from a CBT perspective. These results show how the same changes in behavior during CBT can be due to distinct underlying mechanisms; they predict lower rates of relapse when cognitive interventions focus on inducing uncertainty and on reducing the effects of automatic negative thoughts on behavior.

Simulating lesion-dependent functional recovery mechanisms

Functional recovery after brain damage varies widely and depends on many factors, including lesion site and extent. When a neuronal system is damaged, recovery may occur by engaging residual (e.g., perilesional) components. When damage is extensive, recovery depends on the availability of other intact neural structures that can reproduce the same functional output (i.e., degeneracy). A system's response to damage may occur rapidly, require learning or both. Here, we simulate functional recovery from four different types of lesions, using a generative model of word repetition that comprised a default premorbid system and a less used alternative system. The synthetic lesions (i) completely disengaged the premorbid system, leaving the alternative system intact, (ii) partially damaged both premorbid and alternative systems, and (iii) limited the experience-dependent plasticity of both. The results, across 1000 trials, demonstrate that (i) a complete disconnection of the premorbid system naturally invoked the engagement of the other, (ii) incomplete damage to both systems had a much more devastating long-term effect on model performance and (iii) the effect of reducing learning capacity within each system. These findings contribute to formal frameworks for interpreting the effect of different types of lesions.

Recent advances in the application of predictive coding and active inference models within clinical neuroscience

Research in clinical neuroscience is founded on the idea that a better understanding of brain (dys)function will improve our ability to diagnose and treat neurological and psychiatric disorders. In recent years, neuroscience has converged on the notion that the brain is a 'prediction machine,' in that it actively predicts the sensory input that it will receive if one or another course of action is chosen. These predictions are used to select actions that will (most often, and in the long run) maintain the body within the narrow range of physiological states consistent with survival. This insight has given rise to an area of clinical computational neuroscience research that focuses on characterizing neural circuit architectures that can accomplish these predictive functions, and on how the associated processes may break down or become aberrant within clinical conditions. Here, we provide a brief review of examples of recent work on the application of predictive processing models of brain function to study clinical (psychiatric) disorders, with the aim of highlighting current directions and their potential clinical utility. We offer examples of recent conceptual models, formal mathematical models, and applications of such models in empirical research in clinical populations, with a focus on making this material accessible to clinicians without expertise in computational neuroscience. In doing so, we aim to highlight the potential insights and opportunities that understanding the brain as a prediction machine may offer to clinical research and practice.

Prediction error signaling explains neuronal mismatch responses in the medial prefrontal cortex

The mismatch negativity (MMN) is a key biomarker of automatic deviance detection thought to emerge from 2 cortical sources. First, the auditory cortex (AC) encodes spectral regularities and reports frequency-specific deviances. Then, more abstract representations in the prefrontal cortex (PFC) allow to detect contextual changes of potential behavioral relevance. However, the precise location and time asynchronies between neuronal correlates underlying this frontotemporal network remain unclear and elusive. Our study presented auditory oddball paradigms along with "no-repetition" controls to record mismatch responses in neuronal spiking activity and local field potentials at the rat medial PFC. Whereas mismatch responses in the auditory system are mainly induced by stimulus-dependent effects, we found that auditory responsiveness in the PFC was driven by unpredictability, yielding context-dependent, comparatively delayed, more robust and longer-lasting mismatch responses mostly comprised of prediction error signaling activity. This characteristically different composition discarded that mismatch responses in the PFC could be simply inherited or amplified downstream from the auditory system. Conversely, it is more plausible for the PFC to exert top-down influences on the AC, since the PFC exhibited flexible and potent predictive processing, capable of suppressing redundant input more efficiently than the AC. Remarkably, the time course of the mismatch responses we observed in the spiking activity and local field potentials of the AC and the PFC combined coincided with the time course of the large-scale MMN-like signals reported in the rat brain, thereby linking the microscopic, mesoscopic, and macroscopic levels of automatic deviance detection.

Active inference on discrete state-spaces: A synthesis

Active inference is a normative principle underwriting perception, action, planning, decision-making and learning in biological or artificial agents. From its inception, its associated process theory has grown to incorporate complex generative models, enabling simulation of a wide range of complex behaviours. Due to successive developments in active inference, it is often difficult to see how its underlying principle relates to process theories and practical implementation. In this paper, we try to bridge this gap by providing a complete mathematical synthesis of active inference on discrete state-space models. This technical summary provides an overview of the theory, derives neuronal dynamics from first principles and relates this dynamics to biological processes. Furthermore, this paper provides a fundamental building block needed to understand active inference for mixed generative models; allowing continuous sensations to inform discrete representations. This paper may be used as follows: to guide research towards outstanding challenges, a practical guide on how to implement active inference to simulate experimental behaviour, or a pointer towards various in-silico neurophysiological responses that may be used to make empirical predictions.

Fast fronto-parietal cortical dynamics of conflict detection and context updating in a flanker task

Recent research has found that the traditional target P3 consists of a family of P3-like positivities that can be functionally and topographically dissociated from one another. The current study examined target N2 and P3-like subcomponents indexing conflict detection and context updating at low- and high-order levels in the neural hierarchy during cognitive control. Electroencephalographic signals were recorded from 45 young adults while they completed a hybrid go/nogo flanker task, and Residue Iteration Decomposition (RIDE) was applied to functionally dissociate these peaks. Analyses showed a stimulus-locked frontal N2 revealing early detection and fast perceptual categorization of nogo, congruent and incongruent trials, resulting in frontal P3-like activity elicited by nogo trials in the latency-variable RIDE cluster, and by incongruent trials in the response-locked cluster. The congruent trials did not elicit frontal P3-like activity. These findings suggest that behavioral incongruency effects are related to intermediate and later stages of motor response re-programming.

Green D. Neuromodulatory Control and Language Recovery in Bilingual Aphasia: An Active Inference Approach

Understanding the aetiology of the diverse recovery patterns in bilingual aphasia is a theoretical challenge with implications for treatment. Loss of control over intact language networks provides a parsimonious starting point that can be tested using in-silico lesions. We simulated a complex recovery pattern (alternate antagonism and paradoxical translation) to test the hypothesis-from an established hierarchical control model-that loss of control was mediated by constraints on neuromodulatory resources. We used active (Bayesian) inference to simulate a selective loss of sensory precision; i.e., confidence in the causes of sensations. This in-silico lesion altered the precision of beliefs about task relevant states, including appropriate actions, and reproduced exactly the recovery pattern of interest. As sensory precision has been linked to acetylcholine release, these simulations endorse the conjecture that loss of neuromodulatory control can explain this atypical recovery pattern. We discuss the relevance of this finding for other recovery patterns.

The predictive global neuronal workspace: A formal active inference model of visual consciousness

The global neuronal workspace (GNW) model has inspired over two decades of hypothesis-driven research on the neural basis of consciousness. However, recent studies have reported findings that are at odds with empirical predictions of the model. Further, the macro-anatomical focus of current GNW research has limited the specificity of predictions afforded by the model. In this paper we present a neurocomputational model - based on Active Inference - that captures central architectural elements of the GNW and is able to address these limitations. The resulting 'predictive global workspace' casts neuronal dynamics as approximating Bayesian inference, allowing precise, testable predictions at both the behavioural and neural levels of description. We report simulations demonstrating the model's ability to reproduce: 1) the electrophysiological and behavioural results observed in previous studies of inattentional blindness; and 2) the previously introduced four-way taxonomy predicted by the GNW, which describes the relationship between consciousness, attention, and sensory signal strength. We then illustrate how our model can reconcile/explain (apparently) conflicting findings, extend the GNW taxonomy to include the influence of prior expectations, and inspire novel paradigms to test associated behavioural and neural predictions.

Paradoxical lesions, plasticity and active inference

Paradoxical lesions are secondary brain lesions that ameliorate functional deficits caused by the initial insult. This effect has been explained in several ways; particularly by the reduction of functional inhibition, or by increases in the excitatory-to-inhibitory synaptic balance within perilesional tissue. In this article, we simulate how and when a modification of the excitatory-inhibitory balance triggers the reversal of a functional deficit caused by a primary lesion. For this, we introduce in-silico lesions to an active inference model of auditory word repetition. The first in-silico lesion simulated damage to the extrinsic (between regions) connectivity causing a functional deficit that did not fully resolve over 100 trials of a word repetition task. The second lesion was implemented in the intrinsic (within region) connectivity, compromising the model's ability to rebalance excitatory-inhibitory connections during learning. We found that when the second lesion was mild, there was an increase in experience-dependent plasticity that enhanced performance relative to a single lesion. This paradoxical lesion effect disappeared when the second lesion was more severe because plasticity-related changes were disproportionately amplified in the intrinsic connectivity, relative to lesioned extrinsic connections. Finally, this framework was used to predict the physiological correlates of paradoxical lesions. This formal approach provides new insights into the computational and neurophysiological mechanisms that allow some patients to recover after large or multiple lesions.

Inferring What to Do (And What Not to)

In recent years, the "planning as inference" paradigm has become central to the study of behaviour. The advance offered by this is the formalisation of motivation as a prior belief about "how I am going to act". This paper provides an overview of the factors that contribute to this prior. These are rooted in optimal experimental design, information theory, and statistical decision making. We unpack how these factors imply a functional architecture for motivated behaviour. This raises an important question: how can we put this architecture to work in the service of understanding observed neurobiological structure? To answer this question, we draw from established techniques in experimental studies of behaviour. Typically, these examine the influence of perturbations of the nervous system-which include pathological insults or optogenetic manipulations-to see their influence on behaviour. Here, we argue that the message passing that emerges from inferring what to do can be similarly perturbed. If a given perturbation elicits the same behaviours as a focal brain lesion, this provides a functional interpretation of empirical findings and an anatomical grounding for theoretical results. We highlight examples of this approach that influence different sorts of goal-directed behaviour, active learning, and decision making. Finally, we summarise their implications for the neuroanatomy of inferring what to do (and what not to).

Generalised free energy and active inference

Active inference is an approach to understanding behaviour that rests upon the idea that the brain uses an internal generative model to predict incoming sensory data. The fit between this model and data may be improved in two ways. The brain could optimise probabilistic beliefs about the variables in the generative model (i.e. perceptual inference). Alternatively, by acting on the world, it could change the sensory data, such that they are more consistent with the model. This implies a common objective function (variational free energy) for action and perception that scores the fit between an internal model and the world. We compare two free energy functionals for active inference in the framework of Markov decision processes. One of these is a functional of beliefs (i.e. probability distributions) about states and policies, but a function of observations, while the second is a functional of beliefs about all three. In the former (expected free energy), prior beliefs about outcomes are not part of the generative model (because they are absorbed into the prior over policies). Conversely, in the second (generalised free energy), priors over outcomes become an explicit component of the generative model. When using the free energy function, which is blind to future observations, we equip the generative model with a prior over policies that ensure preferred (i.e. priors over) outcomes are realised. In other words, if we expect to encounter a particular kind of outcome, this lends plausibility to those policies for which this outcome is a consequence. In addition, this formulation ensures that selected policies minimise uncertainty about future outcomes by minimising the free energy expected in the future. When using the free energy functional-that effectively treats future observations as hidden states-we show that policies are inferred or selected that realise prior preferences by minimising the free energy of future expectations. Interestingly, the form of posterior beliefs about policies (and associated belief updating) turns out to be identical under both formulations, but the quantities used to compute them are not.