Neural processes mediating contextual influences on human choice behaviour

The Intersection of Interoception and Anticipation Related to Pain

This chapter focuses on the interconnectedness of bodily awareness, emotion, and time perception, focusing on the roles of interoception and anticipation, pioneered by A.D. (Bud) Craig and advanced by contemporary research. Pain, a pivotal aspect of human experience, serves as a conduit for understanding our relationship with the world. Anticipation of pain, vital for survival, influences subjective pain experiences and is modulated by factors such as physiological reactivity and contextual cues. Emotional states significantly shape pain perception, with chronic pain conditions and affective disorders characterized by dysregulated pain modulation mechanisms. We discuss a state space model for pain, wherein pain functions as a latent construct shaped by both anticipatory and contemporaneous factors. Understanding these mechanisms is critical for informing clinical interventions aimed at pain management and highlights the intersection of interoception, emotion, and pain.

The timescale and direction of influence of a third inferior alternative in human value-learning

The way humans and other animals represent the values of alternatives is context-dependent, as it can be distorted by inferior alternatives that are immediately available for choice (immediate context); or that were encountered in previous episodes (temporal context). Yet, the extent to which the immediate and temporal context (co-) shape context-dependent valuation remains unclear. Here, we asked human participants (onsite: N = 30, online: N = 68) to learn the values associated with three alternatives and explicitly report these values before making binary and ternary choices among the alternatives. We show that context-dependent valuation is evident in the pre-choice value estimates and manifests equally in binary and ternary choices. Accordingly, we conclude that value representations are modulated by the temporal (and not the immediate) context. The direction and across-participants variability of this modulation cannot be captured by extant normalization theories but by a mechanism constructing values through sequential binary comparisons.

Decoding Salience: A Functional Magnetic Resonance Imaging Investigation of Reward and Contextual Unexpectedness in Memory Encoding and Retrieval

The present study investigated the neuromodulatory substrates of salience processing and its impact on memory encoding and behaviour, with a specific focus on two distinct types of salience: reward and contextual unexpectedness. 46 Participants performed a novel task paradigm modulating these two aspects independently and allowing for investigating their distinct and interactive effects on memory encoding while undergoing high-resolution fMRI. By using advanced image processing techniques tailored to examine midbrain and brainstem nuclei with high precision, our study additionally aimed to elucidate differential activation patterns in subcortical nuclei in response to reward-associated and contextually unexpected stimuli, including distinct pathways involving in particular dopaminergic modulation. We observed a differential involvement of the ventral striatum, substantia nigra (SN) and caudate nucleus, as well as a functional specialisation within the subregions of the cingulate cortex for the two salience types. Moreover, distinct subregions within the SN in processing salience could be identified. Dorsal areas preferentially processed salience related to stimulus processing (of both reward and contextual unexpectedness), and ventral areas were involved in salience-related memory encoding (for contextual unexpectedness only). These functional specialisations within SN are in line with different projection patterns of dorsal and ventral SN to brain areas supporting attention and memory, respectively. By disentangling stimulus processing and memory encoding related to two salience types, we hope to further consolidate our understanding of neuromodulatory structures' differential as well as interactive roles in modulating behavioural responses to salient events.

Contextualizing predictive minds

The structure of human memory seems to be optimized for efficient prediction, planning, and behavior. We propose that these capacities rely on a tripartite structure of memory that includes concepts, events, and contexts-three layers that constitute the mental world model. We suggest that the mechanism that critically increases adaptivity and flexibility is the tendency to contextualize. This tendency promotes local, context-encoding abstractions, which focus event- and concept-based planning and inference processes on the task and situation at hand. As a result, cognitive contextualization offers a solution to the frame problem-the need to select relevant features of the environment from the rich stream of sensorimotor signals. We draw evidence for our proposal from developmental psychology and neuroscience. Adopting a computational stance, we present evidence from cognitive modeling research which suggests that context sensitivity is a feature that is critical for maximizing the efficiency of cognitive processes. Finally, we turn to recent deep-learning architectures which independently demonstrate how context-sensitive memory can emerge in a self-organized learning system constrained by cognitively-inspired inductive biases.

Efficient value synthesis in the orbitofrontal cortex explains how loss aversion adapts to the ranges of gain and loss prospects

Is irrational behavior the incidental outcome of biological constraints imposed on neural information processing? In this work, we consider the paradigmatic case of gamble decisions, where gamble values integrate prospective gains and losses. Under the assumption that neurons have a limited firing response range, we show that mitigating the ensuing information loss within artificial neural networks that synthetize value involves a specific form of self-organized plasticity. We demonstrate that the ensuing efficient value synthesis mechanism induces value range adaptation. We also reveal how the ranges of prospective gains and/or losses eventually determine both the behavioral sensitivity to gains and losses and the information content of the network. We test these predictions on two fMRI datasets from the OpenNeuro.org initiative that probe gamble decision-making but differ in terms of the range of gain prospects. First, we show that peoples' loss aversion eventually adapts to the range of gain prospects they are exposed to. Second, we show that the strength with which the orbitofrontal cortex (in particular: Brodmann area 11) encodes gains and expected value also depends upon the range of gain prospects. Third, we show that, when fitted to participant's gambling choices, self-organizing artificial neural networks generalize across gain range contexts and predict the geometry of information content within the orbitofrontal cortex. Our results demonstrate how self-organizing plasticity aiming at mitigating information loss induced by neurons' limited response range may result in value range adaptation, eventually yielding irrational behavior.

Plinko: Eliciting beliefs to build better models of statistical learning and mental model updating

Prior beliefs are central to Bayesian accounts of cognition, but many of these accounts do not directly measure priors. More specifically, initial states of belief heavily influence how new information is assumed to be utilized when updating a particular model. Despite this, prior and posterior beliefs are either inferred from sequential participant actions or elicited through impoverished means. We had participants to play a version of the game 'Plinko', to first elicit individual participant priors in a theoretically agnostic manner. Subsequent learning and updating of participant beliefs was then directly measured. We show that participants hold various priors that cluster around prototypical probability distributions that in turn influence learning. In follow-up studies, we show that participant priors are stable over time and that the ability to update beliefs is influenced by a simple environmental manipulation (i.e., a short break). These data reveal the importance of directly measuring participant beliefs rather than assuming or inferring them as has been widely done in the literature to date. The Plinko game provides a flexible and fecund means for examining statistical learning and mental model updating.

Identification of group differences in predictive anticipatory biasing of pain during uncertainty: preparing for the worst but hoping for the best

ABSTRACT

Pain anticipation during conditions of uncertainty can unveil intrinsic biases, and understanding these biases can guide pain treatment interventions. This study used machine learning and functional magnetic resonance imaging to predict anticipatory responses in a pain anticipation experiment. One hundred forty-seven participants that included healthy controls (n = 57) and individuals with current and/or past mental health diagnosis (n = 90) received cues indicating upcoming pain stimuli: 2 cues predicted high and low temperatures, while a third cue introduced uncertainty. Accurate differentiation of neural patterns associated with specific anticipatory conditions was observed, involving activation in the anterior short gyrus of the insula and the nucleus accumbens. Three distinct response profiles emerged: subjects with a negative bias towards high pain anticipation, those with a positive bias towards low pain anticipation, and individuals whose predictions during uncertainty were unbiased. These profiles remained stable over one year, were consistent across diagnosed psychopathologies, and correlated with cognitive coping styles and underlying insula anatomy. The findings suggest that individualized and stable pain anticipation occurs in uncertain conditions.

Similar functional networks predict performance in both perceptual and value-based decision tasks

There are numerous commonalities between perceptual and preferential decision processes. For instance, previous studies have shown that both of these decision types are influenced by context. Also, the same computational models can explain both. However, the neural processes and functional connections that underlie these similarities between perceptual and value-based decisions are still unclear. Hence, in the current study, we examine whether perceptual and preferential processes can be explained by similar functional networks utilizing data from the Human Connectome Project. We used resting-state functional magnetic resonance imaging data to predict performance of 2 different decision-making tasks: a value-related task (the delay discounting task) and a perceptual task (the flanker task). We then examined the existence of shared predictive-network features across these 2 decision tasks. Interestingly, we found a significant positive correlation between the functional networks, which predicted the value-based and perceptual tasks. In addition, a larger functional connectivity between visual and frontal decision brain areas was a critical feature in the prediction of both tasks. These results demonstrate that functional connections between perceptual and value-related areas in the brain are inherently related to decision-making processes across domains.

Implicit Counterfactual Effect in Partial Feedback Reinforcement Learning: Behavioral and Modeling Approach

Context remarkably affects learning behavior by adjusting option values according to the distribution of available options. Displaying counterfactual outcomes, the outcomes of the unchosen option alongside the chosen one (i.e., providing complete feedback), would increase the contextual effect by inducing participants to compare the two outcomes during learning. However, when the context only consists of the juxtaposition of several options and there is no such explicit counterfactual factor (i.e., only partial feedback is provided), it is not clear whether and how the contextual effect emerges. In this research, we employ Partial and Complete feedback paradigms in which options are associated with different reward distributions. Our modeling analysis shows that the model that uses the outcome of the chosen option for updating the values of both chosen and unchosen options in opposing directions can better account for the behavioral data. This is also in line with the diffusive effect of dopamine on the striatum. Furthermore, our data show that the contextual effect is not limited to probabilistic rewards, but also extends to magnitude rewards. These results suggest that by extending the counterfactual concept to include the effect of the chosen outcome on the unchosen option, we can better explain why there is a contextual effect in situations in which there is no extra information about the unchosen outcome.

A Decision Architecture for Safety Computations

Accurately estimating safety is critical to pursuing nondefensive survival behaviors. However, little attention has been paid to how the human brain computes safety. We conceptualize a model that consists of two components: (i) threat-oriented evaluations that focus on threat value, imminence, and predictability; and (ii) self-oriented evaluations that focus on the agent's experience, strategies, and ability to control the situation. Our model points to the dynamic interaction between these two components as a mechanism of safety estimation. Based on a growing body of human literature, we hypothesize that distinct regions of the ventromedial prefrontal cortex (vmPFC) respond to threat and safety to facilitate survival decisions. We suggest safety is not an inverse of danger, but reflects independent computations that mediate defensive circuits and behaviors.

Prior Reward Conditioning Dampens Hippocampal and Striatal Responses during an Associative Memory Task

Offering reward during encoding typically leads to better memory [Adcock, R. A., Thangavel, A., Whitfield-Gabrieli, S.,Knutson, B., & Gabrieli, J. D. E. Reward-motivated learning: Mesolimbic activation precedes memory formation. Neuron, 50, 507-517, 2006]. Whether such memory benefit persists when tested in a different task context remains, however, largely understudied [Wimmer, G. E., & Buechel, C. Reactivation of reward-related patterns from single past episodes supports memory-based decision making. Journal of Neuroscience, 36, 2868-2880, 2016]. Here, we ask whether reward at encoding leads to a generalized advantage across learning episodes, a question of high importance for any everyday life applications, from education to patient rehabilitation. Although we confirmed that offering monetary reward increased responses in the ventral striatum and pleasantness judgments for pictures used as stimuli, this immediate beneficial effect of reward did not carry over to a subsequent and different picture-location association memory task during which no reward was delivered. If anything, a trend for impaired memory accuracy was observed for the initially high-rewarded pictures as compared to low-rewarded ones. In line with this trend in behavioral performance, fMRI activity in reward (i.e., ventral striatum) and in memory (i.e., hippocampus) circuits was reduced during the encoding of new associations using previously highly rewarded pictures (compared to low-reward pictures). These neural effects extended to new pictures from same, previously highly rewarded semantic category. Twenty-four hours later, delayed recall of associations involving originally highly rewarded items was accompanied by decreased functional connectivity between the hippocampus and two brain regions implicated in value-based learning, the ventral striatum and the ventromedial PFC. We conclude that acquired reward value elicits a downward value-adjustment signal in the human reward circuit when reactivated in a novel nonrewarded context, with a parallel disengagement of memory-reward (hippocampal-striatal) networks, likely to undermine new associative learning. Although reward is known to promote learning, here we show how it may subsequently hinder hippocampal and striatal responses during new associative memory formation.

Quantifying the immediate computational effects of preceding outcomes on subsequent risky choices

Forty years ago, prospect theory introduced the notion that risky options are evaluated relative to their recent context, causing a significant shift in the study of risky monetary decision-making in psychology, economics, and neuroscience. Despite the central role of past experiences, it remains unclear whether, how, and how much past experiences quantitatively influence risky monetary choices moment-to-moment in a nominally learning-free setting. We analyzed a large dataset of risky monetary choices with trial-by-trial feedback to quantify how past experiences, or recent events, influence risky choice behavior and the underlying processes. We found larger recent outcomes both negatively influence subsequent risk-taking and positively influence the weight put on potential losses. Using a hierarchical Bayesian framework to fit a modified version of prospect theory, we demonstrated that the same risks will be evaluated differently given different past experiences. The computations underlying risky decision-making are fundamentally dynamic, even if the environment is not.

Pubertal testosterone correlates with adolescent impatience and dorsal striatal activity

Recent self-report and behavioral studies have demonstrated that pubertal testosterone is related to an increase in risky and impulsive behavior. Yet, the mechanisms underlying such a relationship are poorly understood. Findings from both human and rodent studies point towards distinct striatal pathways including the ventral and dorsal striatum as key target regions for pubertal hormones. In this study we investigated task-related impatience of boys between 10 and 15 years of age (N = 75), using an intertemporal choice task combined with measures of functional magnetic resonance imaging and hormonal assessment. Increased levels of testosterone were associated with a greater response bias towards choosing the smaller sooner option. Furthermore, our results show that testosterone specifically modulates the dorsal, not ventral, striatal pathway. These results provide novel insights into our understanding of adolescent impulsive and risky behaviors and how pubertal hormones are related to neural processes.

Better than expected: the influence of option expectations during decision-making

Our choices often arise from a consideration of options presented in a sequence (e.g. the products in a supermarket row). However, whether the precise sequential order of option presentation affects decision-making remains poorly understood. A recent model of choice proposes that, in a set of options presented sequentially, those that are better than expected will be perceived as more valuable, even when options are objectively equivalent within the set. Inspired by this proposal, we devised a novel decision-making task where we manipulated the order of option presentation together with expectations about option value. Even when we compared trials that were exactly equivalent except for option order, we observed a striking preference for options that were better than expected. Our findings show that expectations about options affect which option will be favoured within a sequence, an influence which is manifested as a preference for better-than-expected options. The findings have potential practical implications, as for example they may help policymakers in devising nudge strategies that rely on ad hoc option orders.

Reference effects on decision-making elicited by previous rewards

Substantial evidence has highlighted reference effects occurring during decision-making, whereby subjective value is not calculated in absolute terms but relative to the distribution of rewards characterizing a context. Among these, within-choice effects are exerted by options simultaneously available during choice. These should be distinguished from between-choice effects, which depend on the distribution of options presented in the past. Influential theories on between-choice effects include Decision-by-Sampling, Expectation-as-Reference and Divisive Normalization. Surprisingly, previous literature has focused on each theory individually disregarding the others. Thus, similarities and differences among theories remain to be systematically examined. Here we fill this gap by offering an overview of the state-of-the-art of research about between-choice reference effects. Our comparison of alternative theories shows that, at present, none of them is able to account for the full range of empirical data. To address this, we propose a model inspired by previous perspectives and based on a logistic framework, hence called logistic model of subjective value. Predictions of the model are analysed in detail about reference effects and risky decision-making. We conclude that our proposal offers a compelling framework for interpreting the multifaceted manifestations of between-choice reference effects.

Reference-point centering and range-adaptation enhance human reinforcement learning at the cost of irrational preferences

In economics and perceptual decision-making contextual effects are well documented, where decision weights are adjusted as a function of the distribution of stimuli. Yet, in reinforcement learning literature whether and how contextual information pertaining to decision states is integrated in learning algorithms has received comparably little attention. Here, we investigate reinforcement learning behavior and its computational substrates in a task where we orthogonally manipulate outcome valence and magnitude, resulting in systematic variations in state-values. Model comparison indicates that subjects' behavior is best accounted for by an algorithm which includes both reference point-dependence and range-adaptation-two crucial features of state-dependent valuation. In addition, we find that state-dependent outcome valuation progressively emerges, is favored by increasing outcome information and correlated with explicit understanding of the task structure. Finally, our data clearly show that, while being locally adaptive (for instance in negative valence and small magnitude contexts), state-dependent valuation comes at the cost of seemingly irrational choices, when options are extrapolated out from their original contexts.

Enhanced motivation of cognitive control in Parkinson's disease

Motor and cognitive deficits in Parkinson's disease (PD) have been argued to reflect motivational deficits. In prior work, however, we have shown that motivation of cognitive control is paradoxically potentiated rather than impaired in Parkinson's disease. This is particularly surprising given the fact that Parkinson's disease is often accompanied by depression, a prototypical disorder of motivation. To replicate our previous finding and assess the effects of depression, we investigated performance of PD patients with (n = 22) and without depression (history) (n = 23) and age‐matched healthy controls (n = 23) on a task specifically designed to measure the effect of reward motivation on task‐switching. We replicated previous findings by showing contrasting effects of reward motivation on task‐switching in PD patients and age‐matched healthy controls. While the promise of high versus low reward improved task‐switching in PD, it tended to impair task‐switching in age‐matched healthy controls. There were no effects of a depression (history) diagnosis in PD patients. These findings reinforce prior observations that Parkinson's disease is accompanied by enhanced incentive motivation of cognitive control and highlight the potential of incentive motivational strategies for overcoming cognitive deficits in Parkinson's disease.

From Believing to Belief: A General Theoretical Model

Cognitive neuroscience research has begun to explore the mental processes underlying what a belief and what believing are. Recent evidence suggests that believing involves fundamental brain functions that result in meaningful probabilistic representations, called beliefs. When relatively stable, these beliefs allow for guidance of behavior in individuals and social groups. However, they are also fluid and can be modified by new relevant information, interpersonal contact, social pressure, and situational demands. We present a theoretical model of believing that can account for the formation of both empirically grounded and metaphysical beliefs.

Multiple timescales of normalized value coding underlie adaptive choice behavior

Adaptation is a fundamental process crucial for the efficient coding of sensory information. Recent evidence suggests that similar coding principles operate in decision-related brain areas, where neural value coding adapts to recent reward history. However, the circuit mechanism for value adaptation is unknown, and the link between changes in adaptive value coding and choice behavior is unclear. Here we show that choice behavior in nonhuman primates varies with the statistics of recent rewards. Consistent with efficient coding theory, decision-making shows increased choice sensitivity in lower variance reward environments. Both the average adaptation effect and across-session variability are explained by a novel multiple timescale dynamical model of value representation implementing divisive normalization. The model predicts empirical variance-driven changes in behavior despite having no explicit knowledge of environmental statistics, suggesting that distributional characteristics can be captured by dynamic model architectures. These findings highlight the importance of treating decision-making as a dynamic process and the role of normalization as a unifying computation for contextual phenomena in choice.

Risk preference and choice stochasticity during decisions for other people

In several contexts, such as finance and politics, people make choices that are relevant for others but irrelevant for oneself. Focusing on decision-making under risk, we compared monetary choices made for one's own interest with choices made on behalf of an anonymous individual. Consistent with the previous literature, other-interest choices were characterized by an increased gambling propensity. We also investigated choice stochasticity, which captures how much decisions vary in similar conditions. An aspect related to choice stochasticity is how much decisions are tuned to the option values, and we found that this was higher during self-interest than during other-interest choices. This effect was observed only in individuals who reported a motivation to distribute rewards unequally, suggesting that it may (at least partially) depend on a motivation to make accurate decisions for others. Our results indicate that, during decision-making under risk, choices for other people are characterized by a decreased tuning to the values of the options, in addition to enhanced risk seeking.

Decreased value-sensitivity in schizophrenia

Pathophysiology in schizophrenia has been linked to aberrant incentive salience, namely the dysfunctional processing of value linked to abnormal dopaminergic activity. In line with this, recent studies showed impaired learning of value in schizophrenia. However, how value is used to guide behaviour independently from learning, as in risky choice, has rarely been examined in this disorder. We studied value-guided choice under risk in patients with schizophrenia and in controls using a task requiring a choice between a certain monetary reward, varying trial-by-trial, and a gamble offering an equal probability of getting double this certain amount or nothing. We observed that patients compared to controls exhibited reduced sensitivity to values, implying that their choices failed to flexibly adapt to the specific values on offer. Moreover, the degree of this value sensitivity inversely correlated with aberrant salience experience, suggesting that the inability to tune choice to value may be a key element of aberrant salience in the illness. Our results help clarify the cognitive mechanisms underlying improper attribution of value in schizophrenia and may thus inform cognitive interventions aimed at reinstating value sensitivity in patients.

Normalized value coding explains dynamic adaptation in the human valuation process

The notion of subjective value is central to choice theories in ecology, economics, and psychology, serving as an integrated decision variable by which options are compared. Subjective value is often assumed to be an absolute quantity, determined in a static manner by the properties of an individual option. Recent neurobiological studies, however, have shown that neural value coding dynamically adapts to the statistics of the recent reward environment, introducing an intrinsic temporal context dependence into the neural representation of value. Whether valuation exhibits this kind of dynamic adaptation at the behavioral level is unknown. Here, we show that the valuation process in human subjects adapts to the history of previous values, with current valuations varying inversely with the average value of recently observed items. The dynamics of this adaptive valuation are captured by divisive normalization, linking these temporal context effects to spatial context effects in decision making as well as spatial and temporal context effects in perception. These findings suggest that adaptation is a universal feature of neural information processing and offer a unifying explanation for contextual phenomena in fields ranging from visual psychophysics to economic choice.

A unifying Bayesian account of contextual effects in value-based choice

Empirical evidence suggests the incentive value of an option is affected by other options available during choice and by options presented in the past. These contextual effects are hard to reconcile with classical theories and have inspired accounts where contextual influences play a crucial role. However, each account only addresses one or the other of the empirical findings and a unifying perspective has been elusive. Here, we offer a unifying theory of context effects on incentive value attribution and choice based on normative Bayesian principles. This formulation assumes that incentive value corresponds to a precision-weighted prediction error, where predictions are based upon expectations about reward. We show that this scheme explains a wide range of contextual effects, such as those elicited by other options available during choice (or within-choice context effects). These include both conditions in which choice requires an integration of multiple attributes and conditions where a multi-attribute integration is not necessary. Moreover, the same scheme explains context effects elicited by options presented in the past or between-choice context effects. Our formulation encompasses a wide range of contextual influences (comprising both within- and between-choice effects) by calling on Bayesian principles, without invoking ad-hoc assumptions. This helps clarify the contextual nature of incentive value and choice behaviour and may offer insights into psychopathologies characterized by dysfunctional decision-making, such as addiction and pathological gambling.

Learning Contextual Reward Expectations for Value Adaptation

Substantial evidence indicates that subjective value is adapted to the statistics of reward expected within a given temporal context. However, how these contextual expectations are learned is poorly understood. To examine such learning, we exploited a recent observation that participants performing a gambling task adjust their preferences as a function of context. We show that, in the absence of contextual cues providing reward information, an average reward expectation was learned from recent past experience. Learning dependent on contextual cues emerged when two contexts alternated at a fast rate, whereas both cue-independent and cue-dependent forms of learning were apparent when two contexts alternated at a slower rate. Motivated by these behavioral findings, we reanalyzed a previous fMRI data set to probe the neural substrates of learning contextual reward expectations. We observed a form of reward prediction error related to average reward such that, at option presentation, activity in ventral tegmental area/substantia nigra and ventral striatum correlated positively and negatively, respectively, with the actual and predicted value of options. Moreover, an inverse correlation between activity in ventral tegmental area/substantia nigra (but not striatum) and predicted option value was greater in participants showing enhanced choice adaptation to context. The findings help understanding the mechanisms underlying learning of contextual reward expectation.

Separate mesocortical and mesolimbic pathways encode effort and reward learning signals

Optimal decision making mandates organisms learn the relevant features of choice options. Likewise, knowing how much effort we should expend can assume paramount importance. A mesolimbic network supports reward learning, but it is unclear whether other choice features, such as effort learning, rely on this same network. Using computational fMRI, we show parallel encoding of effort and reward prediction errors (PEs) within distinct brain regions, with effort PEs expressed in dorsomedial prefrontal cortex and reward PEs in ventral striatum. We show a common mesencephalic origin for these signals evident in overlapping, but spatially dissociable, dopaminergic midbrain regions expressing both types of PE. During action anticipation, reward and effort expectations were integrated in ventral striatum, consistent with a computation of an overall net benefit of a stimulus. Thus, we show that motivationally relevant stimulus features are learned in parallel dopaminergic pathways, with formation of an integrated utility signal at choice.

A Goal-Directed Bayesian Framework for Categorization

Categorization is a fundamental ability for efficient behavioral control. It allows organisms to remember the correct responses to categorical cues and not for every stimulus encountered (hence eluding computational cost or complexity), and to generalize appropriate responses to novel stimuli dependant on category assignment. Assuming the brain performs Bayesian inference, based on a generative model of the external world and future goals, we propose a computational model of categorization in which important properties emerge. These properties comprise the ability to infer latent causes of sensory experience, a hierarchical organization of latent causes, and an explicit inclusion of context and action representations. Crucially, these aspects derive from considering the environmental statistics that are relevant to achieve goals, and from the fundamental Bayesian principle that any generative model should be preferred over alternative models based on an accuracy-complexity trade-off. Our account is a step toward elucidating computational principles of categorization and its role within the Bayesian brain hypothesis.

Active Inference: A Process Theory

This article describes a process theory based on active inference and belief propagation. Starting from the premise that all neuronal processing (and action selection) can be explained by maximizing Bayesian model evidence-or minimizing variational free energy-we ask whether neuronal responses can be described as a gradient descent on variational free energy. Using a standard (Markov decision process) generative model, we derive the neuronal dynamics implicit in this description and reproduce a remarkable range of well-characterized neuronal phenomena. These include repetition suppression, mismatch negativity, violation responses, place-cell activity, phase precession, theta sequences, theta-gamma coupling, evidence accumulation, race-to-bound dynamics, and transfer of dopamine responses. Furthermore, the (approximately Bayes' optimal) behavior prescribed by these dynamics has a degree of face validity, providing a formal explanation for reward seeking, context learning, and epistemic foraging. Technically, the fact that a gradient descent appears to be a valid description of neuronal activity means that variational free energy is a Lyapunov function for neuronal dynamics, which therefore conform to Hamilton's principle of least action.