Active inference and learning

Resilience phenotypes derived from an active inference account of allostasis

Within a theoretical framework of enactive allostasis, we explore active inference strategies for minimizing surprise to achieve resilience in dynamic environments. While individual differences and extrinsic protective factors traditionally account for variability in resilience trajectories following stressor exposure, the enactive model emphasizes the importance of the physical and social environment, specifically the "enactive niche," which is both shaped by and impacts organisms living in it, accounting for variable success in allostatic prediction and accommodation. Enactive allostasis infers or predicts states of the world to minimize surprise and maintain regulation after surprise, i.e., resilience. Action policies are selected in accordance with the inferred state of a dynamic environment; those actions concurrently shape one's environment, buffering against current and potential stressors. Through such inferential construction, multiple potential solutions exist for achieving stability within one's enactive niche. Spanning a range of adaptive resilience strategies, we propose four phenotypes-fragile, durable, resilient, and pro-entropic (PE)-each characterized by a constellation of genetic, epigenetic, developmental, experiential, and environmental factors. Biological regulatory outcomes range from allostatic (over)load in the fragile and durable phenotypes, to allostatic recovery in resilience, and theoretically to increasing allostatic accommodation or "growth" in the proposed PE phenotype. Awareness distinguishes phenotypes by minimizing allostatically demanding surprise and engenders the cognitive and behavioral flexibility empirically associated with resilience. We further propose a role for awareness in proactively shaping one's enactive niche to further minimize surprise. We conclude by exploring the mechanisms of phenotypic plasticity which may bolster individual resilience.

Pathological forgetting from a predictive processing perspective

Recent research suggests that natural forgetting is beneficial, allowing the brain to prioritize relevant information and disregard the irrelevant, thus aiding decision-making and mental health. Conversely, pathological conditions may arise from disruptions in these memory control processes. Without adequate memory control capacities, individuals can suffer from conditions like PTSD or addiction (where unwanted or addiction-related memories persist) on one end of the scale, to conditions such as dementia, Parkinson's disease or traumatic brain injury, which are characterised by heightened rates of forgetting on the other side. This review will explore the concept of predictive processing as a potential mechanism underlying pathological forgetting. It will summarise the neurobiological basis of predictive processing and how it influences what we remember or forget. As evident in the emerging literature, this has distinct implications for understanding pathological forgetting in psychological disorders. Finally, this review will highlight therapeutic interventions that have recently targeted predictive processes and consequently improved symptoms related to forgetting, suggesting translational applications for treatment approaches in these conditions.

Precision-dependent modulation of social attention

Social attention, guided by cues like gaze direction, is crucial for effective social interactions. However, how dynamic environmental context modulates this process remains unclear. Integrating a hierarchical Bayesian model with fMRI, this study investigated how individuals adjusted attention based on the predictions about cue validity (CV). Thirty-three participants performed a modified Posner location-cueing task with varying CV. Behaviorally, individuals' allocation of social attention was finely tuned to the precision (inverse variance) of CV predictions, with the predictions updated by precision-weighted prediction errors (PEs) about the occurrence of target locations. Neuroimaging results revealed that the interaction between allocation of social attention and CV influenced activity in regions involved in spatial attention and/or social perception. Precision-weighted PEs about target locations specifically modulated activity in the temporoparietal junction (TPJ), superior temporal sulcus (STS), and primary visual cortex (V1), underscoring their roles in refining attentional predictions. Dynamic causal modeling (DCM) further demonstrated that enhanced absolute precision-weighted PEs about target locations strengthened the effective connectivity from V1 and STS to TPJ, emphasizing their roles in conveying residual error signals upwards to high-level critical attention areas. These findings emphasized the pivotal role of precision in attentional modulation, enhancing our understanding of context-dependent social attention.

Synechism 2.0: Contours of a new theory of continuity in bioengineering

The methodological principle of synechism, the all-pervading continuity first proposed by Charles Peirce in 1892, is reinvigorated in the present paper to prompt a comprehensive reevaluation of the integrated concepts of life, machines, agency, and intelligence. The evidence comes from the intersections of synthetic bioengineering, developmental biology, and cognitive and computational sciences. As a regulative principle, synechism, "that continuity governs the whole domain of experience in every element of it", has been shown to infiltrate fundamental issues of contemporary biology, including cognition in different substrates, embodied agency, collectives (swarm and nested), intelligence on multiple scales, and developmental bioelectricity in morphogenesis. In the present paper, we make explicit modern biology's turn to this fundamental feature of science in its rejection of conceptual binaries, preference for collectives over individuals, quantitative over qualitative, and multiscale applicability of the emerging hypotheses about the integration of the first principles of the diversity of life. Specifically, synechism presents itself as the bedrock for research encompassing biological machines, chimaeras, organoids, and Xenobots. We then review a synechistic framework that embeds functionalist, information-theoretic, pragmaticist and inferentialist approaches to springboard to continuum-driven biosystemic behaviour.

Mind the semantic gap: semantic efficiency in human computer interfaces

As we become increasingly dependent on technology in our daily lives, the usability of HCIs is a key driver of individual empowerment for us all. A primary focus of AI systems has been to make HCIs easier to use by identifying what users need and agentively taking over some of the cognitive work users would have otherwise performed, as such, they are becoming our delegates. To become effective and reliable delegates, AI agents need to understand all relevant situational semantic context surrounding a user's need and how the tools of the HCI can be leveraged. Current ML systems have fundamental semantic gaps in bespoke human context, real-time world knowledge, and how those relate to HCI tooling. These challenges are difficult to close due factors such as privacy, continual learning, access to real-time context, and how deeply integrated the semantics are with in-context learning. As such, we need to research and explore new ways to safely capture, compactly model, and incrementally evolve semantics in ways that can efficiently integrate into how AI systems act on our behalf. This article presents a thought experiment called the Game of Delegation as a lens to view the effectiveness of delegation and the semantic efficiency with which the delegation was achieved.

The neural correlates of novelty and variability in human decision-making under an active inference framework

Active inference integrates perception, decision-making, and learning into a united theoretical framework, providing an efficient way to trade off exploration and exploitation by minimizing (expected) free energy. In this study, we asked how the brain represents values and uncertainties (novelty and variability), and resolves these uncertainties under the active inference framework in the exploration-exploitation trade-off. Twenty-five participants performed a contextual two-armed bandit task, with electroencephalogram (EEG) recordings. By comparing the model evidence for active inference and reinforcement learning models of choice behavior, we show that active inference better explains human decision-making under novelty and variability, which entails exploration or information seeking. The EEG sensor-level results show that the activity in the frontal, central, and parietal regions is associated with novelty, while the activity in the frontal and central brain regions is associated with variability. The EEG source-level results indicate that the expected free energy is encoded in the frontal pole and middle frontal gyrus and uncertainties are encoded in different brain regions but with overlap. Our study dissociates the expected free energy and uncertainties in active inference theory and their neural correlates, speaking to the construct validity of active inference in characterizing cognitive processes of human decisions. It provides behavioral and neural evidence of active inference in decision processes and insights into the neural mechanism of human decisions under uncertainties.

Sensory attenuation of self-initiated tactile feedback is modulated by stimulus strength and temporal delay in a virtual reality environment

Despite extensive research across various modalities, the precise mechanisms of sensory attenuation (SA) remain debated. Specifically, it remains unclear to what extent SA is influenced by stimulus predictability alone, as opposed to the distinct impact of self-generated actions. Forward models suggest that efference copies of motor commands enable the brain to predict and distinguish anticipated changes in self-initiated sensory input. Predictive processing proposes that predictions about upcoming changes in sensory input are not solely based on efference copies, but rather generated in the form of a generative model integrating external, contextual factors, as well. This study investigated the underlying mechanisms of SA in the tactile domain, specifically examining self-initiation and temporal predictions within a virtual reality (VR) framework. This setup allowed for precise control over sensory feedback in response to movement. Participants (N = 33) engaged in an active condition, moving their hands to elicit a virtual touch. Importantly, visual perception was modified in VR, so that participants touched their rendered-but not physical-hands. The virtual touch triggered the test vibrations on a touch controller (intensities: 0.2, 0.35, 0.5, 0.65, 0.8; in arbitrary units.), the intensity of which was then compared to that of a standard stimulus (intensity: 0.5). In the passive condition, vibrations were presented without movement and were preceded by a visual cue. Further, test vibrations appeared either immediately or after a variable onset delay (700-800ms). Our results revealed a significant effect of the factor "onset delay" on perceived vibration intensity. In addition, we observed interactions between the factors "agency" and "test vibration intensity" and between the factors "agency" and "onset delay," with attenuation effects for immediate vibrations at high intensities and enhancement effects for delayed vibrations at low intensities. These findings emphasize the impact of external, contextual factors and support the notion of a broader, attention-oriented predictive mechanism for the perception of self-initiated stimuli.

Learning of the mean, but not variance, of color distributions cues target location probability

Humans are good at picking up statistical regularities in the environment. Probability cueing paradigms have demonstrated that the location of a target can be predicted based on spatial regularities. This is assumed to rely on flexible spatial priority maps that are influenced by visual context. We investigated whether stimulus features such as color distributions differing in mean and variance can cue location regularities. In experiment 1, participants searched for an oddly colored target diamond in a 6 × 6 set. On each trial, the distractors were drawn from one of two color distributions centered on different color averages. Each distribution was associated with different target location probabilities, one distribution where the target had an 80% chance to appear on the left (the rich location), while the rich location would be on the right for the other distribution. Participants were significantly faster at locating the target when it appeared in the rich location for both distributions, demonstrating learning of the relationship between color average and location probability. In experiments 2 and 3, observers performed a similar search task, but the distributions had different variances with the same average color. There was no evidence that search became faster when the target appeared in a rich location, suggesting that contingencies between target probabilities and color variance were not learned. These results demonstrate how statistical location learning is flexible, with different visual contexts leading to different spatial priority maps, but they also reveal important limits to such learning.

Spontaneous perceptual alternations and higher-order cognitive processes: an exploratory study

The occurrence of spontaneous switches between different interpretations of unchanging, ambiguous stimuli reflects the dynamic nature of unconscious perceptual processing. These perceptual alternations are explained by reciprocal inhibition, where one perception is suppressed while another emerges. The temporal patterns of these shifts vary between people but remain consistent within individuals, potentially reflecting underlying neural and psychological factors. Cognitive flexibility, the ability to switch tasks by inhibiting irrelevant information, may be related to perceptual flexibility. The present study (n = 48) explored the relationship between perceptual dynamics in the Necker Cube and higher-order cognitive processes. Switching rates and perspective durations were correlated with performance on computerized tasks (Stroop Test, Simon Task, and Task Switching Tests) and self-reported scales (Cognitive Flexibility Test, Barratt's Impulsiveness Scale, Depression, Anxiety, and Stress Scale, and Big Five Personality Traits Questionnaire). Results revealed correlations between perceptual dynamics, reaction times, and Cognitive Impulsiveness, suggesting links between perceptual alternation, cognitive processes, and personality traits. Future research should investigate the common mechanisms underlying these processes and investigate causality and temporal dynamics.

Expectation violations signal goals in novel human communication

Communication, often grounded in shared expectations, faces challenges when a Sender and Receiver lack a common linguistic background. Our study explores how people instinctively turn to the fundamental principles of the physical world to overcome such barriers. Specifically, through an experimental game in which Senders convey messages via trajectories, we investigate how they develop novel strategies without relying on common linguistic cues. We build a computational model based on the principle of expectancy violations and a set of common universal priors derived from movement kinetics. The model replicates participant-designed messages with high accuracy and shows how its core variable-surprise-predicts the Receiver's physiological and neuronal responses in brain areas processing expectation violations. This work highlights the adaptability of human communication, showing how surprise can be a powerful tool in forming new communicative strategies without relying on common language.

Gesture sonification for enhancing agency: an exploratory study on healthy participants

Background

Body awareness (BA) and proprioception, which are essential components of the sense of agency (SA), are often altered in various mental disorders such as posttraumatic stress disorder (PTSD). However, the relationship between BA, proprioception, and SA, as well as the methods to manipulate them, remain unclear. This study explored using real-time gesture sonification (GS), i.e., wearable technology transforming body movements into sounds, to enhance proprioception, BA, and thus the SA.

Methods

In this within-subjects design, 17 healthy adults (mean age = 25.5 years) with varying dance expertise (novice, amateur, expert) improvised movements to match sounds with and without auditory feedback from motion sensors on wrists/ankles modulated by their gestures. BA, immersion, pleasure, and self-efficacy were measured.

Results

Sonification significantly increased body awareness, reward, and immersion (all p < 0.05).

Conclusion

GS can enhance BA and the SA, pleasure, and control during physical activity. This highlights potential mental health applications, such as agency-based therapies for PTSD. Manipulating bodily perception could improve symptoms and embodiment. Further research should replicate this in clinical populations and explore neurocognitive mechanisms.

Shifting boundaries, extended minds: ambient technology and extended allostatic control

This article applies the thesis of the extended mind to ambient smart environments. These systems are characterised by an environment, such as a home or classroom, infused with multiple, highly networked streams of smart technology working in the background, learning about the user and operating without an explicit interface or any intentional sensorimotor engagement from the user. We analyse these systems in the context of work on the "classical" extended mind, characterised by conditions such as "trust and glue" and phenomenal transparency, and find that these conditions are ill-suited to describing our engagement with ambient smart environments. We then draw from the active inference framework, a theory of brain function which casts cognition as a process of embodied uncertainty minimisation, to develop a version of the extended mind grounded in a process ontology, where the boundaries of mind are understood to be multiple and always shifting. Given this more fluid account of the extended mind, we argue that ambient smart environments should be thought of as extended allostatic control systems, operating more or less invisibly to support an agent's biological capacity for minimising uncertainty over multiple, interlocking timescales. Thus, we account for the functionality of ambient smart environments as extended systems, and in so doing, utilise a markedly different version of the classical thesis of extended mind.

As One and Many: Relating Individual and Emergent Group-Level Generative Models in Active Inference

Active inference under the Free Energy Principle has been proposed as an across-scales compatible framework for understanding and modelling behaviour and self-maintenance. Crucially, a collective of active inference agents can, if they maintain a group-level Markov blanket, constitute a larger group-level active inference agent with a generative model of its own. This potential for computational scale-free structures speaks to the application of active inference to self-organizing systems across spatiotemporal scales, from cells to human collectives. Due to the difficulty of reconstructing the generative model that explains the behaviour of emergent group-level agents, there has been little research on this kind of multi-scale active inference. Here, we propose a data-driven methodology for characterising the relation between the generative model of a group-level agent and the dynamics of its constituent individual agents. We apply methods from computational cognitive modelling and computational psychiatry, applicable for active inference as well as other types of modelling approaches. Using a simple Multi-Armed Bandit task as an example, we employ the new ActiveInference.jl library for Julia to simulate a collective of agents who are equipped with a Markov blanket. We use sampling-based parameter estimation to make inferences about the generative model of the group-level agent, and we show that there is a non-trivial relationship between the generative models of individual agents and the group-level agent they constitute, even in this simple setting. Finally, we point to a number of ways in which this methodology might be applied to better understand the relations between nested active inference agents across scales.

Exploring when to exploit: the cognitive underpinnings of foraging-type decisions in relation to psychopathy

Impairments in reinforcement learning (RL) might underlie the tendency of individuals with elevated psychopathic traits to behave exploitatively, as they fail to learn from their mistakes. Most studies on the topic have focused on binary choices, while everyday functioning requires us to learn the value of multiple options. In this study, we evaluated the cognitive correlates of naturalistic foraging-type decision-making and their electrophysiological signatures in a community sample (n = 108) with varying degrees of psychopathic traits. Reinforcers with different salience were included in a foraging-type decision-making task. Recruitment of various cognitive processes was estimated with a computational model and electrophysiology, and the relationships to psychopathic traits were assessed. Higher Antisocial traits were associated with a bias towards expecting more volatility in the environment when high-salience reinforcers were used. Additionally, higher levels of Interpersonal traits were associated with reduced learning from personalized rewards, as evidenced by reductions in the prediction errors (PEs) about rate of change. Higher Affective traits were associated with lower PEs and aberrant learning from painful punishments. Lastly, the PEs about rate of change were reflected in the trial-wise trajectories of Feedback-Related Negativity event-related potentials. Together, our results point to the importance of volatility processing in understanding aberrant decision-making in relation to psychopathy, demonstrate the relationships between psychopathic traits and learning through reward and punishment, and emphasise the potentially more beneficial effect of personalized rewards and punishment for improving reinforcement-based decision-making in individuals with elevated psychopathic traits.

Modelling sensory attenuation as Bayesian causal inference across two datasets

INTRODUCTION

To interact with the environment, it is crucial to distinguish between sensory information that is externally generated and inputs that are self-generated. The sensory consequences of one's own movements tend to induce attenuated behavioral- and neural responses compared to externally generated inputs. We propose a computational model of sensory attenuation (SA) based on Bayesian Causal Inference, where SA occurs when an internal cause for sensory information is inferred.

METHODS

Experiment 1investigates sensory attenuation during a stroking movement. Tactile stimuli on the stroking finger were suppressed, especially when they were predictable. Experiment 2 showed impaired delay detection between an arm movement and a video of the movement when participants were moving vs. when their arm was moved passively. We reconsider these results from the perspective of Bayesian Causal Inference (BCI). Using a hierarchical Markov Model (HMM) and variational message passing, we first qualitatively capture patterns of task behavior and sensory attenuation in simulations. Next, we identify participant-specific model parameters for both experiments using optimization.

RESULTS

A sequential BCI model is well equipped to capture empirical patterns of SA across both datasets. Using participant-specific optimized model parameters, we find a good agreement between data and model predictions, with the model capturing both tactile detections in Experiment 1 and delay detections in Experiment 2.

DISCUSSION

BCI is an appropriate framework to model sensory attenuation in humans. Computational models of sensory attenuation may help to bridge the gap across different sensory modalities and experimental paradigms and may contribute towards an improved description and understanding of deficits in specific patient groups (e.g. schizophrenia).

Introducing ActiveInference.jl: A Julia Library for Simulation and Parameter Estimation with Active Inference Models

We introduce a new software package for the Julia programming language, the library ActiveInference.jl. To make active inference agents with Partially Observable Markov Decision Process (POMDP) generative models available to the growing research community using Julia, we re-implemented the pymdp library for Python. ActiveInference.jl is compatible with cutting-edge Julia libraries designed for cognitive and behavioural modelling, as it is used in computational psychiatry, cognitive science and neuroscience. This means that POMDP active inference models can now be easily fit to empirically observed behaviour using sampling, as well as variational methods. In this article, we show how ActiveInference.jl makes building POMDP active inference models straightforward, and how it enables researchers to use them for simulation, as well as fitting them to data or performing a model comparison.

Flow and intuition: a systems neuroscience comparison

This paper explores the relationship between intuition and flow from a neurodynamics perspective. Flow and intuition represent two cognitive phenomena rooted in nonconscious information processing; however, there are clear differences in both their phenomenal characteristics and, more broadly, their contribution to action and cognition. We propose, extrapolating from dual processing theory, that intuition serves as a rapid, nonconscious decision-making process, while flow facilitates this process in action, achieving optimal cognitive control and performance without [conscious] deliberation. By exploring these points of convergence between flow and intuition, we also attempt to reconcile the apparent paradox of the presence of enhanced intuition in flow, which is also a state of heightened cognitive control. To do so, we utilize a revised dual-processing framework, which allows us to productively align and differentiate flow and intuition (including intuition in flow). Furthermore, we draw on recent work examining flow from an active inference perspective. Our account not only heightens understanding of human cognition and consciousness, but also raises new questions for future research, aiming to deepen our comprehension of how flow and intuition can be harnessed to elevate human performance and wellbeing.

The role of brain oscillatory activity in processing the informative value of feedback during rule acquisition

Information conveyed through feedback enables individuals to learn new routines and better adapt to their environment. However, the neural mechanisms of rule-related information of feedback have not been fully elucidated. Herein, we quantified the effect of informative value on feedback via a rule induction task (RIT), in which participants were required to find the correct sorting rule based on feedback. To disengage the effects of informative value and valence on feedback in the RIT, a control task was developed in which feedback only involved the valence aspect and no reference for subsequent selections. We measured power and intertrial phase clustering (ITPC) values via EEG to determine the neural mechanisms of rule-related feedback. The results revealed that (1) differences in oscillatory activities between positive and negative feedback were only observed during the control task, and no such effect was found in the RIT task. This finding suggests that the participants paid more attention to rule-related information than to the correctness of feedback during rule learning. (2) The task differences under positive or negative feedback were associated with the delta-theta and alpha-beta bands, and this pattern was similar within the frontal and parietal regions. These findings suggest that the processing of rule-related information of feedback relies on broad frequency bands within the frontoparietal cortex to facilitate rule information integration. In summary, these findings indicate that multiple frequency bands are involved in encoding the informative value aspect of feedback, and individuals rely on this aspect of feedback rather than valence during rule learning.

Body as First Teacher: The Role of Rhythmic Visceral Dynamics in Early Cognitive Development

Embodied cognition-the idea that mental states and processes should be understood in relation to one's bodily constitution and interactions with the world-remains a controversial topic within cognitive science. Recently, however, increasing interest in predictive processing theories among proponents and critics of embodiment alike has raised hopes of a reconciliation. This article sets out to appraise the unificatory potential of predictive processing, focusing in particular on embodied formulations of active inference. Our analysis suggests that most active-inference accounts invoke weak, potentially trivial conceptions of embodiment; those making stronger claims do so independently of the theoretical commitments of the active-inference framework. We argue that a more compelling version of embodied active inference can be motivated by adopting a diachronic perspective on the way rhythmic physiological activity shapes neural development in utero. According to this visceral afferent training hypothesis, early-emerging physiological processes are essential not only for supporting the biophysical development of neural structures but also for configuring the cognitive architecture those structures entail. Focusing in particular on the cardiovascular system, we propose three candidate mechanisms through which visceral afferent training might operate: (a) activity-dependent neuronal development, (b) periodic signal modeling, and (c) oscillatory network coordination.

Influences of speaking task demands on sensorimotor oscillations in adults who stutter: Implications for speech motor control

OBJECTIVE

Motivated by previous inconsistent findings, this study aims to improve understanding of sensorimotor beta (β; 15-30 Hz) and alpha (α; 8-14 Hz) speech-related power differences between stuttering and non-stuttering adults.

METHODS

Electroencephalography was recorded as adults who stutter (AWS) and matched fluent controls answered questions in Quiet and Informational Masked backgrounds. Bilateral sensorimotor β and α power during speech planning and execution were measured from mu (μ) rhythm components.

RESULTS

Compared to controls, AWS exhibited reduced left hemisphere β and α power in both speaking conditions during speech planning and execution. AWS displayed reduced left α power in the Informational Masking compared to Quiet. Within AWS β and α power, which were tightly coupled, oppositely predicted stuttering severity and β-α dissociation (β minus α) was the strongest predictor.

CONCLUSION

Neither β nor α power are reliable markers of speech motor stability due to their sensitivity to speech task automaticity. However, relationships between these two sensorimotor rhythms warrant further investigation for understanding motor control.

SIGNIFICANCE

Data help explain previous mixed findings in reference to extant models of speech motor control in stuttering and may have clinical implications for developing neurostimulation protocols targeting improved speech fluency.

Contextualizing predictive minds

The structure of human memory seems to be optimized for efficient prediction, planning, and behavior. We propose that these capacities rely on a tripartite structure of memory that includes concepts, events, and contexts-three layers that constitute the mental world model. We suggest that the mechanism that critically increases adaptivity and flexibility is the tendency to contextualize. This tendency promotes local, context-encoding abstractions, which focus event- and concept-based planning and inference processes on the task and situation at hand. As a result, cognitive contextualization offers a solution to the frame problem-the need to select relevant features of the environment from the rich stream of sensorimotor signals. We draw evidence for our proposal from developmental psychology and neuroscience. Adopting a computational stance, we present evidence from cognitive modeling research which suggests that context sensitivity is a feature that is critical for maximizing the efficiency of cognitive processes. Finally, we turn to recent deep-learning architectures which independently demonstrate how context-sensitive memory can emerge in a self-organized learning system constrained by cognitively-inspired inductive biases.

Learning dynamic cognitive map with autonomous navigation

Inspired by animal navigation strategies, we introduce a novel computational model to navigate and map a space rooted in biologically inspired principles. Animals exhibit extraordinary navigation prowess, harnessing memory, imagination, and strategic decision-making to traverse complex and aliased environments adeptly. Our model aims to replicate these capabilities by incorporating a dynamically expanding cognitive map over predicted poses within an active inference framework, enhancing our agent's generative model plasticity to novelty and environmental changes. Through structure learning and active inference navigation, our model demonstrates efficient exploration and exploitation, dynamically expanding its model capacity in response to anticipated novel un-visited locations and updating the map given new evidence contradicting previous beliefs. Comparative analyses in mini-grid environments with the clone-structured cognitive graph model (CSCG), which shares similar objectives, highlight our model's ability to rapidly learn environmental structures within a single episode, with minimal navigation overlap. Our model achieves this without prior knowledge of observation and world dimensions, underscoring its robustness and efficacy in navigating intricate environments.

Mechanisms of brain self-regulation: psychological factors, mechanistic models and neural substrates

While neurofeedback represents a promising tool for neuroscience and a brain self-regulation approach to psychological rehabilitation, the field faces several problems and challenges. Current research has shown great variability and even failure among human participants in learning to self-regulate target features of brain activity with neurofeedback. A better understanding of cognitive mechanisms, psychological factors and neural substrates underlying self-regulation might help improve neurofeedback's scientific and clinical practices. This article reviews the current understanding of the neural mechanisms of brain self-regulation by drawing on findings from human and animal studies in neurofeedback, brain-computer/machine interfaces and neuroprosthetics. In this article, we look closer at the following topics: cognitive processes and psychophysiological factors affecting self-regulation, theoretical models and neural substrates underlying self-regulation, and finally, we provide an outlook on the outstanding gaps in knowledge and technical challenges. This article is part of the theme issue 'Neurofeedback: new territories and neurocognitive mechanisms of endogenous neuromodulation'.

Nature heals: An informational entropy account of self-organization and change in field psychotherapy

This paper reviews biophysical models of psychotherapeutic change based on synergetics and the free energy principle. These models suggest that introducing sensory surprise into the patient-therapist system can lead to self-organization and the formation of new attractor states, disrupting entrenched patterns of thoughts, emotions, and behaviours. We propose that the therapist can facilitate this process by cultivating epistemic trust and modulating embodied attention to allow surprising affective states to enter shared awareness. Transient increases in free energy enable the update of generative models, expanding the range of experiences available within the patient-therapist phenomenal field. We hypothesize that patterns of disorganization at behavioural and physiological levels, indexed by increased entropy, complexity, and lower determinism, are key markers and predictors of psychotherapeutic gains. Future research should investigate how the therapist's openness to novelty shapes therapeutic outcomes.

Survival in a world of complex dangers

How did our nomadic ancestors continually adapt to the seemingly limitless and unpredictable number of dangers in the natural world? We argue that human defensive behaviors are dynamically constructed to facilitate survival in capricious and itinerant environments. We first hypothesize that internal and external states result in state constructions that combine to form a meta-representation. When a threat is detected, it triggers the action construction. Action constructions are formed through two contiguous survival strategies: generalization strategies, which are used when encountering new threats and ecologies. Generalization strategies are associated with cognitive representations that have high dimensionality and which furnish flexible psychological constructs, including relations between threats, and imagination, and which converge through the construction of defensive states. We posit that generalization strategies drive 'explorative' behaviors including information seeking, where the goal is to increase knowledge that can be used to mitigate current and future threats. Conversely, specialization strategies entail lower dimensional representations, which underpin specialized, sometimes reflexive, or habitual survival behaviors that are 'exploitative'. Together, these strategies capture a central adaptive feature of human survival systems: self-preservation in response to a myriad of threats.

Is artificial consciousness achievable? Lessons from the human brain

We here analyse the question of developing artificial consciousness from an evolutionary perspective, taking the evolution of the human brain and its relation with consciousness as a reference model or as a benchmark. This kind of analysis reveals several structural and functional features of the human brain that appear to be key for reaching human-like complex conscious experience and that current research on Artificial Intelligence (AI) should take into account in its attempt to develop systems capable of human-like conscious processing. We argue that, even if AI is limited in its ability to emulate human consciousness for both intrinsic (i.e., structural and architectural) and extrinsic (i.e., related to the current stage of scientific and technological knowledge) reasons, taking inspiration from those characteristics of the brain that make human-like conscious processing possible and/or modulate it, is a potentially promising strategy towards developing conscious AI. Also, it cannot be theoretically excluded that AI research can develop partial or potentially alternative forms of consciousness that are qualitatively different from the human form, and that may be either more or less sophisticated depending on the perspectives. Therefore, we recommend neuroscience-inspired caution in talking about artificial consciousness: since the use of the same word "consciousness" for humans and AI becomes ambiguous and potentially misleading, we propose to clearly specify which level and/or type of consciousness AI research aims to develop, as well as what would be common versus differ in AI conscious processing compared to human conscious experience.

Associative Learning and Active Inference

Associative learning is a behavioral phenomenon in which individuals develop connections between stimuli or events based on their co-occurrence. Initially studied by Pavlov in his conditioning experiments, the fundamental principles of learning have been expanded on through the discovery of a wide range of learning phenomena. Computational models have been developed based on the concept of minimizing reward prediction errors. The Rescorla-Wagner model, in particular, is a well-known model that has greatly influenced the field of reinforcement learning. However, the simplicity of these models restricts their ability to fully explain the diverse range of behavioral phenomena associated with learning. In this study, we adopt the free energy principle, which suggests that living systems strive to minimize surprise or uncertainty under their internal models of the world. We consider the learning process as the minimization of free energy and investigate its relationship with the Rescorla-Wagner model, focusing on the informational aspects of learning, different types of surprise, and prediction errors based on beliefs and values. Furthermore, we explore how well-known behavioral phenomena such as blocking, overshadowing, and latent inhibition can be modeled within the active inference framework. We accomplish this by using the informational and novelty aspects of attention, which share similar ideas proposed by seemingly contradictory models such as Mackintosh and Pearce-Hall models. Thus, we demonstrate that the free energy principle, as a theoretical framework derived from first principles, can integrate the ideas and models of associative learning proposed based on empirical experiments and serve as a framework for a better understanding of the computational processes behind associative learning in the brain.

Learning-based object's stiffness and shape estimation with confidence level in multi-fingered hand grasping

Introduction

When humans grasp an object, they are capable of recognizing its characteristics, such as its stiffness and shape, through the sensation of their hands. They can also determine their level of confidence in the estimated object properties. In this study, we developed a method for multi-fingered hands to estimate both physical and geometric properties, such as the stiffness and shape of an object. Their confidence levels were measured using proprioceptive signals, such as joint angles and velocity.

Method

We have developed a learning framework based on probabilistic inference that does not necessitate hyperparameters to maintain equilibrium between the estimation of diverse types of properties. Using this framework, we have implemented recurrent neural networks that estimate the stiffness and shape of grasped objects with their uncertainty in real time.

Results

We demonstrated that the trained neural networks are capable of representing the confidence level of estimation that includes the degree of uncertainty and task difficulty in the form of variance and entropy.

Discussion

We believe that this approach will contribute to reliable state estimation. Our approach would also be able to combine with flexible object manipulation and probabilistic inference-based decision making.

Active Inference and Deep Generative Modeling for Cognitive Ultrasound

Ultrasound (US) has the unique potential to offer access to medical imaging to anyone, everywhere. Devices have become ultraportable and cost-effective, akin to the stethoscope. Nevertheless, and despite many advances, US image quality and diagnostic efficacy are still highly operator- and patient-dependent. In difficult-to-image patients, image quality is often insufficient for reliable diagnosis. In this article, we put forth the idea that US imaging systems can be recast as information-seeking agents that engage in reciprocal interactions with their anatomical environment. Such agents autonomously adapt their transmit-receive sequences to fully personalize imaging and actively maximize information gain in situ. To that end, we will show that the sequence of pulse-echo experiments that a US system performs can be interpreted as a perception-action loop: the action is the data acquisition, probing tissue with acoustic waves and recording reflections at the detection array, and perception is the inference of the anatomical and or functional state, potentially including associated diagnostic quantities. We then equip systems with a mechanism to actively reduce uncertainty and maximize diagnostic value across a sequence of experiments, treating action and perception jointly using Bayesian inference given generative models of the environment and action-conditional pulse-echo observations. Since the representation capacity of the generative models dictates both the quality of inferred anatomical states and the effectiveness of inferred sequences of future imaging actions, we will be greatly leveraging the enormous advances in deep generative modeling (generative AI), which are currently disrupting many fields and society at large. Finally, we show some examples of cognitive, closed-loop, US systems that perform active beamsteering and adaptive scanline selection based on deep generative models that track anatomical belief states.

Supervised structure learning

This paper concerns structure learning or discovery of discrete generative models. It focuses on Bayesian model selection and the assimilation of training data or content, with a special emphasis on the order in which data are ingested. A key move-in the ensuing schemes-is to place priors on the selection of models, based upon expected free energy. In this setting, expected free energy reduces to a constrained mutual information, where the constraints inherit from priors over outcomes (i.e., preferred outcomes). The resulting scheme is first used to perform image classification on the MNIST dataset to illustrate the basic idea, and then tested on a more challenging problem of discovering models with dynamics, using a simple sprite-based visual disentanglement paradigm and the Tower of Hanoi (cf., blocks world) problem. In these examples, generative models are constructed autodidactically to recover (i.e., disentangle) the factorial structure of latent states-and their characteristic paths or dynamics.

Deconstructing Deep Active Inference: A Contrarian Information Gatherer

Active inference is a theory of perception, learning, and decision making that can be applied to neuroscience, robotics, psychology, and machine learning. Recently, intensive research has been taking place to scale up this framework using Monte Carlo tree search and deep learning. The goal of this activity is to solve more complicated tasks using deep active inference. First, we review the existing literature and then progressively build a deep active inference agent as follows: we (1) implement a variational autoencoder (VAE), (2) implement a deep hidden Markov model (HMM), and (3) implement a deep critical hidden Markov model (CHMM). For the CHMM, we implemented two versions, one minimizing expected free energy, CHMM[EFE] and one maximizing rewards, CHMM[reward]. Then we experimented with three different action selection strategies: the ε-greedy algorithm as well as softmax and best action selection. According to our experiments, the models able to solve the dSprites environment are the ones that maximize rewards. On further inspection, we found that the CHMM minimizing expected free energy almost always picks the same action, which makes it unable to solve the dSprites environment. In contrast, the CHMM maximizing reward keeps on selecting all the actions, enabling it to successfully solve the task. The only difference between those two CHMMs is the epistemic value, which aims to make the outputs of the transition and encoder networks as close as possible. Thus, the CHMM minimizing expected free energy repeatedly picks a single action and becomes an expert at predicting the future when selecting this action. This effectively makes the KL divergence between the output of the transition and encoder networks small. Additionally, when selecting the action down the average reward is zero, while for all the other actions, the expected reward will be negative. Therefore, if the CHMM has to stick to a single action to keep the KL divergence small, then the action down is the most rewarding. We also show in simulation that the epistemic value used in deep active inference can behave degenerately and in certain circumstances effectively lose, rather than gain, information. As the agent minimizing EFE is not able to explore its environment, the appropriate formulation of the epistemic value in deep active inference remains an open question.

Linking actions and memories: Probing the interplay of action-effect congruency, agency experience, and recognition memory

Adult humans experience agency when their action causes certain events (sense of agency). Moreover, they can later remember what these events were (memory). Here, we investigate how the relationship between actions and events shapes agency experience and memory for the corresponding events. Participants performed actions that produced stimuli that were either congruent or incongruent to the action while memory of these stimuli was probed in a recognition test. Additionally, predictability of the effect was manipulated in Experiment 1 by using either randomly interleaved or blocked ordering of action-congruent and action-incongruent events. In Experiment 2, the size of the action space was manipulated by allowing participants to choose between three or six possible responses. The results indicated a heightened sense of agency following congruent compared to incongruent trials, with this effect being increased given a larger available action space, as well as a greater sense of agency given higher predictability of the effect. Recognition memory was better for stimuli presented in congruent compared to incongruent trials, with no discernible effects of effect predictability or the size of the action space. The results point towards a joint influence of predictive and postdictive processes on agency experience and suggest a link between control and memory. The partial dissociation of influences on agency experience and memory cast doubt on a mediating role of agency experience on the relationship between action-effect congruency and memory. Theoretical accounts for this relationship are discussed.

Active inference goes to school: the importance of active learning in the age of large language models

Human learning essentially involves embodied interactions with the material world. But our worlds now include increasing numbers of powerful and (apparently) disembodied generative artificial intelligence (AI). In what follows we ask how best to understand these new (somewhat 'alien', because of their disembodied nature) resources and how to incorporate them in our educational practices. We focus on methodologies that encourage exploration and embodied interactions with 'prepared' material environments, such as the carefully organized settings of Montessori education. Using the active inference framework, we approach our questions by thinking about human learning as epistemic foraging and prediction error minimization. We end by arguing that generative AI should figure naturally as new elements in prepared learning environments by facilitating sequences of precise prediction error enabling trajectories of self-correction. In these ways, we anticipate new synergies between (apparently) disembodied and (essentially) embodied forms of intelligence. This article is part of the theme issue 'Minds in movement: embodied cognition in the age of artificial intelligence'.

A Computational Framework for Understanding the Impact of Prior Experiences on Pain Perception and Neuropathic Pain

Pain perception is influenced not only by sensory input from afferent neurons but also by cognitive factors such as prior expectations. It has been suggested that overly precise priors may be a key contributing factor to chronic pain states such as neuropathic pain. However, it remains an open question how overly precise priors in favor of pain might arise. Here, we first verify that a Bayesian approach can describe how statistical integration of prior expectations and sensory input results in pain phenomena such as placebo hypoalgesia, nocebo hyperalgesia, chronic pain, and spontaneous neuropathic pain. Our results indicate that the value of the prior, which is determined by the internal model parameters, may be a key contributor to these phenomena. Next, we apply a hierarchical Bayesian approach to update the parameters of the internal model based on the difference between the predicted and the perceived pain, to reflect that people integrate prior experiences in their future expectations. In contrast with simpler approaches, this hierarchical model structure is able to show for placebo hypoalgesia and nocebo hyperalgesia how these phenomena can arise from prior experiences in the form of a classical conditioning procedure. We also demonstrate the phenomenon of offset analgesia, in which a disproportionally large pain decrease is obtained following a minor reduction in noxious stimulus intensity. Finally, we turn to simulations of neuropathic pain, where our hierarchical model corroborates that persistent non-neuropathic pain is a risk factor for developing neuropathic pain following denervation, and additionally offers an interesting prediction that complete absence of informative painful experiences could be a similar risk factor. Taken together, these results provide insight to how prior experiences may contribute to pain perception, in both experimental and neuropathic pain, which in turn might be informative for improving strategies of pain prevention and relief.

Evaluating the explanatory power of the Conscious Turing Machine

The recent "Conscious Turing Machine" (CTM) proposal offered by Manuel and Lenore Blum aims to define and explore consciousness, contribute to the solution of the hard problem, and demonstrate the value of theoretical computer science with respect to the study of consciousness. Surprisingly, given the ambitiousness and novelty of the proposal (and the prominence of its creators), CTM has received relatively little attention. We here seek to remedy this by offering an exhaustive evaluation of CTM. Our evaluation considers the explanatory power of CTM in three different domains of interdisciplinary consciousness studies: the philosophy of mind, cognitive neuroscience, and computation. Based on our evaluation in each of the target domains, at present, any claim that CTM constitutes progress is premature. Nevertheless, the model has potential, and we highlight several possible avenues of future research which proponents of the model may pursue in its development.

The cerebellum and the Mirror Neuron System: A matter of inhibition? From neurophysiological evidence to neuromodulatory implications. A narrative review

Mirror neurons show activity during both the execution (AE) and observation of actions (AO). The Mirror Neuron System (MNS) could be involved during motor imagery (MI) as well. Extensive research suggests that the cerebellum is interconnected with the MNS and may be critically involved in its activities. We gathered evidence on the cerebellum's role in MNS functions, both theoretically and experimentally. Evidence shows that the cerebellum plays a major role during AO and MI and that its lesions impair MNS functions likely because, by modulating the activity of cortical inhibitory interneurons with mirror properties, the cerebellum may contribute to visuomotor matching, which is fundamental for shaping mirror properties. Indeed, the cerebellum may strengthen sensory-motor patterns that minimise the discrepancy between predicted and actual outcome, both during AE and AO. Furthermore, through its connections with the hippocampus, the cerebellum might be involved in internal simulations of motor programs during MI. Finally, as cerebellar neuromodulation might improve its impact on MNS activity, we explored its potential neurophysiological and neurorehabilitation implications.

Prior Context and Individual Alpha Frequency Influence Predictive Processing during Language Comprehension

The extent to which the brain predicts upcoming information during language processing remains controversial. To shed light on this debate, the present study reanalyzed Nieuwland and colleagues' (2018) [Nieuwland, M. S., Politzer-Ahles, S., Heyselaar, E., Segaert, K., Darley, E., Kazanina, N., et al. Large-scale replication study reveals a limit on probabilistic prediction in language comprehension. eLife, 7, e33468, 2018] replication of DeLong and colleagues (2015) [DeLong, K. A., Urbach, T. P., & Kutas, M. Probabilistic word pre-activation during language comprehension inferred from electrical brain activity. Nature Neuroscience, 8, 1117-1121, 2005]. Participants (n = 356) viewed sentences containing articles and nouns of varying predictability, while their EEG was recorded. We measured ERPs preceding the critical words (namely, the semantic prediction potential), in conjunction with postword N400 patterns and individual neural metrics. ERP activity was compared with two measures of word predictability: cloze probability and lexical surprisal. In contrast to prior literature, semantic prediction potential amplitudes did not increase as cloze probability increased, suggesting that the component may not reflect prediction during natural language processing. Initial N400 results at the article provided evidence against phonological prediction in language, in line with Nieuwland and colleagues' findings. Strikingly, however, when the surprisal of the prior words in the sentence was included in the analysis, increases in article surprisal were associated with increased N400 amplitudes, consistent with prediction accounts. This relationship between surprisal and N400 amplitude was not observed when the surprisal of the two prior words was low, suggesting that expectation violations at the article may be overlooked under highly predictable conditions. Individual alpha frequency also modulated the relationship between article surprisal and the N400, emphasizing the importance of individual neural factors for prediction. The present study extends upon existing neurocognitive models of language and prediction more generally, by illuminating the flexible and subject-specific nature of predictive processing.

An Integrated theory of false insights and beliefs under psychedelics

Psychedelics are recognised for their potential to re-orient beliefs. We propose a model of how psychedelics can, in some cases, lead to false insights and thus false beliefs. We first review experimental work on laboratory-based false insights and false memories. We then connect this to insights and belief formation under psychedelics using the active inference framework. We propose that subjective and brain-based alterations caused by psychedelics increases the quantity and subjective intensity of insights and thence beliefs, including false ones. We offer directions for future research in minimising the risk of false and potentially harmful beliefs arising from psychedelics. Ultimately, knowing how psychedelics may facilitate false insights and beliefs is crucial if we are to optimally leverage their therapeutic potential.

'Snakes and ladders' in paleoanthropology: From cognitive surprise to skillfulness a million years ago

A paradigmatic account may suffice to explain behavioral evolution in early Homo. We propose a parsimonious account that (1) could explain a particular, frequently-encountered, archeological outcome of behavior in early Homo - namely, the fashioning of a Paleolithic stone 'handaxe' - from a biological theoretic perspective informed by the free energy principle (FEP); and that (2) regards instances of the outcome as postdictive or retrodictive, circumstantial corroboration. Our proposal considers humankind evolving as a self-organizing biological ecosystem at a geological time-scale. We offer a narrative treatment of this self-organization in terms of the FEP. Specifically, we indicate how 'cognitive surprises' could underwrite an evolving propensity in early Homo to express sporadic unorthodox or anomalous behavior. This co-evolutionary propensity has left us a legacy of Paleolithic artifacts that is reminiscent of a 'snakes and ladders' board game of appearances, disappearances, and reappearances of particular archeological traces of Paleolithic behavior. When detected in the Early and Middle Pleistocene record, anthropologists and archeologists often imagine evidence of unusual or novel behavior in terms of early humankind ascending the rungs of a figurative phylogenetic 'ladder' - as if these corresponded to progressive evolution of cognitive abilities that enabled incremental achievements of increasingly innovative technical prowess, culminating in the cognitive ascendancy of Homo sapiens. The conjecture overlooks a plausible likelihood that behavior by an individual who was atypical among her conspecifics could have been disregarded in a community of Hominina (for definition see Appendix 1) that failed to recognize, imagine, or articulate potential advantages of adopting hitherto unorthodox behavior. Such failure, as well as diverse fortuitous demographic accidents, would cause exceptional personal behavior to be ignored and hence unremembered. It could disappear by a pitfall, down a 'snake', as it were, in the figurative evolutionary board game; thereby causing a discontinuity in the evolution of human behavior that presents like an evolutionary puzzle. The puzzle discomforts some paleoanthropologists trained in the natural and life sciences. They often dismiss it, explaining it away with such self-justifying conjectures as that, maybe, separate paleospecies of Homo differentially possessed different cognitive abilities, which, supposedly, could account for the presence or absence in the Pleistocene archeological record of traces of this or that behavioral outcome or skill. We argue that an alternative perspective - that inherits from the FEP and an individual's 'active inference' about its surroundings and of its own responses - affords a prosaic, deflationary, and parsimonious way to account for appearances, disappearances, and reappearances of particular behavioral outcomes and skills of early humankind.

Joint modeling of choices and reaction times based on Bayesian contextual behavioral control

In cognitive neuroscience and psychology, reaction times are an important behavioral measure. However, in instrumental learning and goal-directed decision making experiments, findings often rely only on choice probabilities from a value-based model, instead of reaction times. Recent advancements have shown that it is possible to connect value-based decision models with reaction time models. However, typically these models do not provide an integrated account of both value-based choices and reaction times, but simply link two types of models. Here, we propose a novel integrative joint model of both choices and reaction times by combining a computational account of Bayesian sequential decision making with a sampling procedure. This allows us to describe how internal uncertainty in the planning process shapes reaction time distributions. Specifically, we use a recent context-specific Bayesian forward planning model which we extend by a Markov chain Monte Carlo (MCMC) sampler to obtain both choices and reaction times. As we will show this makes the sampler an integral part of the decision making process and enables us to reproduce, using simulations, well-known experimental findings in value based-decision making as well as classical inhibition and switching tasks. Specifically, we use the proposed model to explain both choice behavior and reaction times in instrumental learning and automatized behavior, in the Eriksen flanker task and in task switching. These findings show that the proposed joint behavioral model may describe common underlying processes in these different decision making paradigms.

Predictive Processing During Cue-Outcome Associative Learning in Autistic Children

PURPOSE

Predictive coding theories posit that autism is characterized by an over-adjustment to prediction errors, resulting in frequent updates of prior beliefs. Atypical weighting of prediction errors is generally considered to negatively impact the construction of stable models of the world, but may also yield beneficial effects. In a novel associative learning paradigm, we investigated whether unexpected events trigger faster learning updates in favour of subtle but fully predictive cues in autistic children compared to their non-autistic counterparts. We also explored the relationship between children's language proficiency and their predictive performances.

METHODS

Anticipatory fixations and explicit predictions were recorded during three associative learning tasks with deterministic or probabilistic contingencies. One of the probabilistic tasks was designed so that a fully predictive but subtle cue was overshadowed by a less predictive salient one.

RESULTS

Both autistic and non-autistic children based their learning on the salient cue, and, contrary to our predictions, showed no signs of updating in favour of the subtle cue. While both groups demonstrated associative learning, autistic children made less accurate explicit predictions than their non-autistic peers in all tasks. Explicit prediction performances were positively correlated with language proficiency in non-autistic children, but no such correlation was observed in autistic children.

CONCLUSION

These results suggest no over-adjustment to prediction errors in autistic children and highlight the need to control for general performance in cue-outcome associative learning in predictive processing studies. Further research is needed to explore the nature of the relationship between predictive processing and language development in autism.

The Utility of Heartrate and Heartrate Variability Biofeedback for the Improvement of Interoception across Behavioural, Physiological and Neural Outcome Measures: A Systematic Review

Interoceptive dysfunctions are increasingly implicated in a number of physical and mental health conditions. Accordingly, there is a pertinent need for therapeutic interventions which target interoceptive deficits. Heartrate and heartrate variability biofeedback therapy (HR(V)-BF), interventions which train individuals to regulate their cardiovascular signals and constrain these within optimal parameters through breathing, could enhance the functioning of interoceptive pathways via stimulation of the vagus nerve. Consequently, this narrative systematic review sought to synthesise the current state of the literature with regard to the potential of HR(V)-BF as an interoceptive intervention across behavioural, physiological and neural outcome measures related to interoception. In total, 77 papers were included in this review, with the majority using physiological outcome measures. Overall, findings were mixed with respect to improvements in the outcome measures after HR(V)-BF. However, trends suggested that effects on measures related to interoception were stronger when resonance frequency breathing and an intense treatment protocol were employed. Based on these findings, we propose a three-stage model by which HR(V)-BF may improve interoception which draws upon principles of interoceptive inference and predictive coding. Furthermore, we provide specific directions for future research, which will serve to advance the current knowledge state.

Forgetting ourselves in flow: an active inference account of flow states and how we experience ourselves within them

Flow has been described as a state of optimal performance, experienced universally across a broad range of domains: from art to athletics, gaming to writing. However, its phenomenal characteristics can, at first glance, be puzzling. Firstly, individuals in flow supposedly report a loss of self-awareness, even though they perform in a manner which seems to evince their agency and skill. Secondly, flow states are felt to be effortless, despite the prerequisite complexity of the tasks that engender them. In this paper, we unpick these features of flow, as well as others, through the active inference framework, which posits that action and perception are forms of active Bayesian inference directed at sustained self-organisation; i.e., the minimisation of variational free energy. We propose that the phenomenology of flow is rooted in the deployment of high precision weight over (i) the expected sensory consequences of action and (ii) beliefs about how action will sequentially unfold. This computational mechanism thus draws the embodied cognitive system to minimise the ensuing (i.e., expected) free energy through the exploitation of the pragmatic affordances at hand. Furthermore, given the challenging dynamics the flow-inducing situation presents, attention must be wholly focussed on the unfolding task whilst counterfactual planning is restricted, leading to the attested loss of the sense of self-as-object. This involves the inhibition of both the sense of self as a temporally extended object and higher-order, meta-cognitive forms of self-conceptualisation. Nevertheless, we stress that self-awareness is not entirely lost in flow. Rather, it is pre-reflective and bodily. Our approach to bodily-action-centred phenomenology can be applied to similar facets of seemingly agentive experience beyond canonical flow states, providing insights into the mechanisms of so-called selfless experiences, embodied expertise and wellbeing.

Reach-to-Grasp and tactile discrimination task: A new task for the study of sensory-motor learning

Although active touch in rodents arises from the forepaws as well as whiskers, most research on active touch only focuses on whiskers. This results in a paucity of tasks designed to assess the process of active touch with a forepaw. We develop a new experimental task, the Reach-to-Grasp and Tactile Discrimination task (RGTD task), to examine active touch with a forepaw in rodents, particularly changes in processes of active touch during motor skill learning. In the RGTD task, animals are required to (1) extend their forelimb to an object, (2) grasp the object, and (3) manipulate the grasped object with the forelimb. The animals must determine the direction of the manipulation based on active touch sensations arising during the period of the grasping. In experiment 1 of the present study, we showed that rats can learn the RGTD task. In experiment 2, we confirmed that the rats are capable of reversal learning of the RGTD task. The RGTD task shared most of the reaching movements involved with conventional forelimb reaching tasks. From the standpoint of a discrimination task, the RGTD task enables rigorous experimental control, for example by removing bias in the stimulus-response correspondence, and makes it possible to utilize diverse experimental procedures that have been difficult in prior tasks.

Synergizing habits and goals with variational Bayes

Behaving efficiently and flexibly is crucial for biological and artificial embodied agents. Behavior is generally classified into two types: habitual (fast but inflexible), and goal-directed (flexible but slow). While these two types of behaviors are typically considered to be managed by two distinct systems in the brain, recent studies have revealed a more sophisticated interplay between them. We introduce a theoretical framework using variational Bayesian theory, incorporating a Bayesian intention variable. Habitual behavior depends on the prior distribution of intention, computed from sensory context without goal-specification. In contrast, goal-directed behavior relies on the goal-conditioned posterior distribution of intention, inferred through variational free energy minimization. Assuming that an agent behaves using a synergized intention, our simulations in vision-based sensorimotor tasks explain the key properties of their interaction as observed in experiments. Our work suggests a fresh perspective on the neural mechanisms of habits and goals, shedding light on future research in decision making.

Neuron-level Prediction and Noise can Implement Flexible Reward-Seeking Behavior

We show that neural networks can implement reward-seeking behavior using only local predictive updates and internal noise. These networks are capable of autonomous interaction with an environment and can switch between explore and exploit behavior, which we show is governed by attractor dynamics. Networks can adapt to changes in their architectures, environments, or motor interfaces without any external control signals. When networks have a choice between different tasks, they can form preferences that depend on patterns of noise and initialization, and we show that these preferences can be biased by network architectures or by changing learning rates. Our algorithm presents a flexible, biologically plausible way of interacting with environments without requiring an explicit environmental reward function, allowing for behavior that is both highly adaptable and autonomous. Code is available at https://github.com/ccli3896/PaN.

Active Inference With Empathy Mechanism for Socially Behaved Artificial Agents in Diverse Situations

This article proposes a method for an artificial agent to behave in a social manner. Although defining proper social behavior is difficult because it differs from situation to situation, the agent following the proposed method adaptively behaves appropriately in each situation by empathizing with the surrounding others. The proposed method is achieved by incorporating empathy into active inference. We evaluated the proposed method regarding control of autonomous mobile robots in diverse situations. From the evaluation results, an agent controlled by the proposed method could behave more adaptively socially than an agent controlled by the standard active inference in the diverse situations. In the case of two agents, the agent controlled with the proposed method behaved in a social way that reduced the other agent's travel distance by 13.7% and increased the margin between the agents by 25.8%, even though it increased the agent's travel distance by 8.2%. Also, the agent controlled with the proposed method behaved more socially when it was surrounded by altruistic others but less socially when it was surrounded by selfish others.

Imagination vs. routines: festive time, weekly time, and the predictive brain

This paper examines the relationship between societal structures shaped by traditions, norms, laws, and customs, and creative expressions in arts and media through the lens of the predictive coding framework in cognitive science. The article proposes that both dimensions of culture can be viewed as adaptations designed to enhance and train the brain's predictive abilities in the social domain. Traditions, norms, laws, and customs foster shared predictions and expectations among individuals, thereby reducing uncertainty in social environments. On the other hand, arts and media expose us to simulated experiences that explore alternative social realities, allowing the predictive machinery of the brain to hone its skills through exposure to a wider array of potentially relevant social circumstances and scenarios. We first review key principles of predictive coding and active inference, and then explore the rationale of cultural traditions and artistic culture in this perspective. Finally, we draw parallels between institutionalized normative habits that stabilize social worlds and creative and imaginative acts that temporarily subvert established conventions to inject variability.