How usefulness shapes neural representations during goal-directed behavior

The geometry of efficient codes: How rate-distortion trade-offs distort the latent representations of generative models

Living organisms rely on internal models of the world to act adaptively. These models, because of resource limitations, cannot encode every detail and hence need to compress information. From a cognitive standpoint, information compression can manifest as a distortion of latent representations, resulting in the emergence of representations that may not accurately reflect the external world or its geometry. Rate-distortion theory formalizes the optimal way to compress information while minimizing such distortions, by considering factors such as capacity limitations, the frequency and the utility of stimuli. However, while this theory explains why the above factors distort latent representations, it does not specify which specific distortions they produce. To address this question, here we investigate how rate-distortion trade-offs shape the latent representations of images in generative models, specifically Beta Variational Autoencoders ([Formula: see text]-VAEs), under varying constraints of model capacity, data distributions, and task objectives. By systematically exploring these factors, we identify three primary distortions in latent representations: prototypization, specialization, and orthogonalization. These distortions emerge as signatures of information compression, reflecting the model's adaptation to capacity limitations, data imbalances, and task demands. Additionally, our findings demonstrate that these distortions can coexist, giving rise to a rich landscape of latent spaces, whose geometry could differ significantly across generative models subject to different constraints. Our findings contribute to explain how the normative constraints of rate-distortion theory shape the geometry of latent representations of generative models of artificial systems and living organisms.

The human reward system encodes the subjective value of ideas during creative thinking

Creative thinking involves the evaluation of one's ideas in order to select the best one, but the cognitive and neural mechanisms underlying this evaluation remain unclear. Using a combination of creativity and rating tasks, this study demonstrates that individuals attribute subjective values to their ideas, as a relative balance of their originality and adequacy. This relative balance depends on individual preferences and predicts individuals' creative abilities. Using functional Magnetic Resonance Imaging, we find that the Default Mode and the Executive Control Networks respectively encode the originality and adequacy of ideas, and that the human reward system encodes their subjective value. Interestingly, the relative functional connectivity of the Default Mode and Executive Control Networks with the human reward system correlates with the relative balance of adequacy and originality in individuals' preferences. These results add valuation to the incomplete behavioral and neural accounts of creativity, offering perspectives on the influence of individual preferences on creative abilities.

Context changes retrieval of prospective outcomes during decision deliberation

Foreseeing the future outcomes is the art of decision-making. Substantial evidence shows that, during choice deliberation, the brain can retrieve prospective decision outcomes. However, decisions are seldom made in a vacuum. Context carries information that can radically affect the outcomes of a choice. Nevertheless, most investigations of retrieval processes examined decisions in isolation, disregarding the context in which they occur. Here, we studied how context shapes prospective outcome retrieval during deliberation. We designed a decision-making task where participants were presented with object-context pairs and made decisions which led to a certain outcome. We show during deliberation, likely outcomes were retrieved in transient patterns of neural activity, as early as 3 s before participants decided. The strength of prospective outcome retrieval explains participants' behavioral efficiency, but only when context affects the decision outcome. Our results suggest context imparts strong constraints on retrieval processes and how neural representations are shaped during decision-making.

Representational spaces in orbitofrontal and ventromedial prefrontal cortex: task states, values, and beyond

The orbitofrontal cortex (OFC) and ventromedial-prefrontal cortex (vmPFC) play a key role in decision-making and encode task states in addition to expected value. We review evidence suggesting a connection between value and state representations and argue that OFC / vmPFC integrate stimulus, context, and outcome information. Comparable encoding principles emerge in late layers of deep reinforcement learning (RL) models, where single nodes exhibit similar forms of mixed-selectivity, which enables flexible readout of relevant variables by downstream neurons. Based on these lines of evidence, we suggest that outcome-maximization leads to complex representational spaces that are insufficiently characterized by linear value signals that have been the focus of most prior research on the topic. Major outstanding questions concern the role of OFC/ vmPFC in learning across tasks, in encoding of task-irrelevant aspects, and the role of hippocampus-PFC interactions.

Quality space computations for consciousness

The quality space hypothesis about conscious experience proposes that conscious sensory states are experienced in relation to other possible sensory states. For instance, the colour red is experienced as being more like orange, and less like green or blue. Recent empirical findings suggest that subjective similarity space can be explained in terms of similarities in neural activation patterns. Here, we consider how localist, workspace, and higher-order theories of consciousness can accommodate claims about the qualitative character of experience and functionally support a quality space. We review existing empirical evidence for each of these positions, and highlight novel experimental tools, such as altering local activation spaces via brain stimulation or behavioural training, that can distinguish these accounts.

Uncovering the determinants of brain functioning, behavior and their interplay in the light of context

Notwithstanding the huge progress in molecular and cellular neuroscience, our ability to understand the brain and develop effective treatments promoting mental health is still limited. This can be partially ascribed to the reductionist, deterministic and mechanistic approaches in neuroscience that struggle with the complexity of the central nervous system. Here, I introduce the Context theory of constrained systems proposing a novel role of contextual factors and genetic, molecular and neural substrates in determining brain functioning and behavior. This theory entails key conceptual implications. First, context is the main driver of behavior and mental states. Second, substrates, from genes to brain areas, have no direct causal link to complex behavioral responses as they can be combined in multiple ways to produce the same response and different responses can impinge on the same substrates. Third, context and biological substrates play distinct roles in determining behavior: context drives behavior, substrates constrain the behavioral repertoire that can be implemented. Fourth, since behavior is the interface between the central nervous system and the environment, it is a privileged level of control and orchestration of brain functioning. Such implications are illustrated through the Kitchen metaphor of the brain. This theoretical framework calls for the revision of key concepts in neuroscience and psychiatry, including causality, specificity and individuality. Moreover, at the clinical level, it proposes treatments inducing behavioral changes through contextual interventions as having the highest impact to reorganize the complexity of the human mind and to achieve a long-lasting improvement in mental health.

Goal commitment is supported by vmPFC through selective attention

When striking a balance between commitment to a goal and flexibility in the face of better options, people often demonstrate strong goal perseveration. Here, using functional MRI (n = 30) and lesion patient (n = 26) studies, we argue that the ventromedial prefrontal cortex (vmPFC) drives goal commitment linked to changes in goal-directed selective attention. Participants performed an incremental goal pursuit task involving sequential decisions between persisting with a goal versus abandoning progress for better alternative options. Individuals with stronger goal perseveration showed higher goal-directed attention in an interleaved attention task. Increasing goal-directed attention also affected abandonment decisions: while pursuing a goal, people lost their sensitivity to valuable alternative goals while remaining more sensitive to changes in the current goal. In a healthy population, individual differences in both commitment biases and goal-oriented attention were predicted by baseline goal-related activity in the vmPFC. Among lesion patients, vmPFC damage reduced goal commitment, leading to a performance benefit.

Functional specialization of medial and lateral orbitofrontal cortex in inferential decision-making

Inferring prospective outcomes and updating behavior are prerequisites for making flexible decisions in the changing world. These abilities are highly associated with the functions of the orbitofrontal cortex (OFC) in humans and animals. The functional specialization of OFC subregions in decision-making has been established in animals. However, the understanding of how human OFC contributes to decision-making remains limited. Therefore, we studied this issue by examining the information representation and functional interactions of human OFC subregions during inference-based decision-making. We found that the medial OFC (mOFC) and lateral OFC (lOFC) collectively represented the inferred outcomes which, however, were context-general coding in the mOFC and context-specific in the lOFC. Furthermore, the mOFC-motor and lOFC-frontoparietal functional connectivity may indicate the motor execution of mOFC and the cognitive control of lOFC during behavioral updating. In conclusion, our findings support the dissociable functional roles of OFC subregions in decision-making.

The computational structure of consummatory anhedonia

Anhedonia is a reduction in enjoyment, motivation, or interest. It is common across mental health disorders and a harbinger of poor treatment outcomes. The enjoyment aspect, termed 'consummatory anhedonia', in particular poses fundamental questions about how the brain constructs rewards: what processes determine how intensely a reward is experienced? Here, we outline limitations of existing computational conceptualisations of consummatory anhedonia. We then suggest a richer reinforcement learning (RL) account of consummatory anhedonia with a reconceptualisation of subjective hedonic experience in terms of goal progress. This accounts qualitatively for the impact of stress, dysfunctional cognitions, and maladaptive beliefs on hedonic experience. The model also offers new views on the treatments for anhedonia.

The online metacognitive control of decisions

Difficult decisions typically involve mental effort, which scales with the deployment of cognitive (e.g., mnesic, attentional) resources engaged in processing decision-relevant information. But how does the brain regulate mental effort? A possibility is that the brain optimizes a resource allocation problem, whereby the amount of invested resources balances its expected cost (i.e. effort) and benefit. Our working assumption is that subjective decision confidence serves as the benefit term of the resource allocation problem, hence the "metacognitive" nature of decision control. Here, we present a computational model for the online metacognitive control of decisions or oMCD. Formally, oMCD is a Markov Decision Process that optimally solves the ensuing resource allocation problem under agnostic assumptions about the inner workings of the underlying decision system. We demonstrate how this makes oMCD a quasi-optimal control policy for a broad class of decision processes, including -but not limited to- progressive attribute integration. We disclose oMCD's main properties (in terms of choice, confidence and response time), and show that they reproduce most established empirical results in the field of value-based decision making. Finally, we discuss the possible connections between oMCD and most prominent neurocognitive theories about decision control and mental effort regulation.

Rationality, preferences, and emotions with biological constraints: it all starts from our senses

Is the role of our sensory systems to represent the physical world as accurately as possible? If so, are our preferences and emotions, often deemed irrational, decoupled from these 'ground-truth' sensory experiences? We show why the answer to both questions is 'no'. Brain function is metabolically costly, and the brain loses some fraction of the information that it encodes and transmits. Therefore, if brains maximize objective functions that increase the fitness of their species, they should adapt to the objective-maximizing rules of the environment at the earliest stages of sensory processing. Consequently, observed 'irrationalities', preferences, and emotions stem from the necessity for our early sensory systems to adapt and process information while considering the metabolic costs and internal states of the organism.

Naturalistic reinforcement learning

Humans possess a remarkable ability to make decisions within real-world environments that are expansive, complex, and multidimensional. Human cognitive computational neuroscience has sought to exploit reinforcement learning (RL) as a framework within which to explain human decision-making, often focusing on constrained, artificial experimental tasks. In this article, we review recent efforts that use naturalistic approaches to determine how humans make decisions in complex environments that better approximate the real world, providing a clearer picture of how humans navigate the challenges posed by real-world decisions. These studies purposely embed elements of naturalistic complexity within experimental paradigms, rather than focusing on simplification, generating insights into the processes that likely underpin humans' ability to navigate complex, multidimensional real-world environments so successfully.

Sex-based differences in fairness norm compliance and neural circuitry

Human behavior often aligns with fairness norms, either voluntarily or under external pressure, like sanctions. Prior research has identified distinct neural activation patterns associated with voluntary and sanction-based compliance or non-compliance with fairness norms. However, an investigation gap exists into potential neural connectivity patterns and sex-based differences. To address this, we conducted a study using a monetary allocation game and functional magnetic resonance imaging to examine how neural activity and connectivity differ between sexes across three norm compliance conditions: voluntary, sanction-based, and voluntary post-sanctions. Fifty-five adults (27 females) participated, revealing that punishment influenced decisions, leading to strategic calculations and reduced generosity in voluntary compliance post-sanctions. Moreover, there were sex-based differences in neural activation and connectivity across the different compliance conditions. Specifically, the connectivity between the right dorsolateral prefrontal cortex and right dorsal anterior insular appeared to mediate intuitive preferences, with variations across norm compliance conditions and sexes. These findings imply potential sex-based differences in intuitive motivation for diverse norm compliance conditions. Our insights contribute to a better understanding of the neural pathways involved in fairness norm compliance and clarify sex-based differences, offering implications for future investigations into psychiatric and neurological disorders characterized by atypical socialization and mentalizing.

A goal-centric outlook on learning

Goals play a central role in human cognition. However, computational theories of learning and decision-making often take goals as given. Here, we review key empirical findings showing that goals shape the representations of inputs, responses, and outcomes, such that setting a goal crucially influences the central aspects of any learning process: states, actions, and rewards. We thus argue that studying goal selection is essential to advance our understanding of learning. By following existing literature in framing goal selection within a hierarchy of decision-making problems, we synthesize important findings on the principles underlying goal value attribution and exploration strategies. Ultimately, we propose that a goal-centric perspective will help develop more complete accounts of learning in both biological and artificial agents.

Goal-Dependent Hippocampal Representations Facilitate Self-Control

Hippocampal activity linking past experiences and simulations of the future with current goals can play an important role in decision-making. The representation of information within the hippocampus may be especially critical in situations where one needs to overcome past rewarding experiences and exert self-control. Self-control success or failure may depend on how information is represented in the hippocampus and how effectively the representation process can be modified to achieve a specific goal. We test this hypothesis using representational similarity analyses of human (female/male) neuroimaging data during a dietary self-control task in which individuals must overcome taste temptations to choose healthy foods. We find that self-control is indeed associated with the way individuals represent taste information (valance) in the hippocampus and how taste representations there adapt to align with different goals/contexts. Importantly, individuals who were able to shift their hippocampal representations to a larger degree to align with the current motivation were better able to exert self-control when facing a dietary challenge. These results suggest an alternative or complementary neurobiological pathway leading to self-control success and indicate the need to update the classical view of self-control to continue to advance our understanding of its behavioral and neural underpinnings.SIGNIFICANCE STATEMENT The paper provides a new perspective on what leads to successful self-control at the behavioral and neurobiological levels. Our data suggest that self-control is enhanced when individuals adjust hippocampal processing to align with current goals.

Intrinsic rewards explain context-sensitive valuation in reinforcement learning

When observing the outcome of a choice, people are sensitive to the choice's context, such that the experienced value of an option depends on the alternatives: getting $1 when the possibilities were 0 or 1 feels much better than when the possibilities were 1 or 10. Context-sensitive valuation has been documented within reinforcement learning (RL) tasks, in which values are learned from experience through trial and error. Range adaptation, wherein options are rescaled according to the range of values yielded by available options, has been proposed to account for this phenomenon. However, we propose that other mechanisms-reflecting a different theoretical viewpoint-may also explain this phenomenon. Specifically, we theorize that internally defined goals play a crucial role in shaping the subjective value attributed to any given option. Motivated by this theory, we develop a new "intrinsically enhanced" RL model, which combines extrinsically provided rewards with internally generated signals of goal achievement as a teaching signal. Across 7 different studies (including previously published data sets as well as a novel, preregistered experiment with replication and control studies), we show that the intrinsically enhanced model can explain context-sensitive valuation as well as, or better than, range adaptation. Our findings indicate a more prominent role of intrinsic, goal-dependent rewards than previously recognized within formal models of human RL. By integrating internally generated signals of reward, standard RL theories should better account for human behavior, including context-sensitive valuation and beyond.

Frontopolar cortex represents complex features and decision value during choice between environments

Important decisions often involve choosing between complex environments that define future item encounters. Despite its importance for adaptive behavior and distinct computational challenges, decision-making research primarily focuses on item choice, ignoring environment choice altogether. Here we contrast previously studied item choice in ventromedial prefrontal cortex with lateral frontopolar cortex (FPl) linked to environment choice. Furthermore, we propose a mechanism for how FPl decomposes and represents complex environments during decision making. Specifically, we trained a choice-optimized, brain-naive convolutional neural network (CNN) and compared predicted CNN activation with actual FPl activity. We showed that the high-dimensional FPl activity decomposes environment features to represent the complexity of an environment to make such choice possible. Moreover, FPl functionally connects with posterior cingulate cortex for guiding environment choice. Further probing FPl's computation revealed a parallel processing mechanism in extracting multiple environment features.

Task state representations in vmPFC mediate relevant and irrelevant value signals and their behavioral influence

The ventromedial prefrontal-cortex (vmPFC) is known to contain expected value signals that inform our choices. But expected values even for the same stimulus can differ by task. In this study, we asked how the brain flexibly switches between such value representations in a task-dependent manner. Thirty-five participants alternated between tasks in which either stimulus color or motion predicted rewards. We show that multivariate vmPFC signals contain a rich representation that includes the current task state or context (motion/color), the associated expected value, and crucially, the irrelevant value of the alternative context. We also find that irrelevant value representations in vmPFC compete with relevant value signals, interact with task-state representations and relate to behavioral signs of value competition. Our results shed light on vmPFC's role in decision making, bridging between its role in mapping observations onto the task states of a mental map, and computing expected values for multiple states.

Sensory perception relies on fitness-maximizing codes

Sensory information encoded by humans and other organisms is generally presumed to be as accurate as their biological limitations allow. However, perhaps counterintuitively, accurate sensory representations may not necessarily maximize the organism's chances of survival. To test this hypothesis, we developed a unified normative framework for fitness-maximizing encoding by combining theoretical insights from neuroscience, computer science, and economics. Behavioural experiments in humans revealed that sensory encoding strategies are flexibly adapted to promote fitness maximization, a result confirmed by deep neural networks with information capacity constraints trained to solve the same task as humans. Moreover, human functional MRI data revealed that novel behavioural goals that rely on object perception induce efficient stimulus representations in early sensory structures. These results suggest that fitness-maximizing rules imposed by the environment are applied at early stages of sensory processing in humans and machines.

Continuously changing memories: a framework for proactive and non-linear consolidation

The traditional view of long-term memory is that memory traces mature in a predetermined 'linear' process: their neural substrate shifts from rapidly plastic medial temporal regions towards stable neocortical networks. We propose that memories remain malleable, not by repeated reinstantiations of this linear process but instead via dynamic routes of proactive and non-linear consolidation: memories change, their trajectory is flexible and reversible, and their physical basis develops continuously according to anticipated demands. Studies demonstrating memory updating, increasing hippocampal dependence to support adaptive use, and rapid neocortical plasticity provide evidence for continued non-linear consolidation. Although anticipated demand can affect all stages of memory formation, the extent to which it shapes the physical memory trace repeatedly and proactively will require further dedicated research.

Goals, usefulness and abstraction in value-based choice

Colombian drug lord Pablo Escobar, while on the run, purportedly burned two million dollars in banknotes to keep his daughter warm. A stark reminder that, in life, circumstances and goals can quickly change, forcing us to reassess and modify our values on-the-fly. Studies in decision-making and neuroeconomics have often implicitly equated value to reward, emphasising the hedonic and automatic aspect of the value computation, while overlooking its functional (concept-like) nature. Here we outline the computational and biological principles that enable the brain to compute the usefulness of an option or action by creating abstractions that flexibly adapt to changing goals. We present different algorithmic architectures, comparing ideas from artificial intelligence (AI) and cognitive neuroscience with psychological theories and, when possible, drawing parallels.

Readiness to remember: predicting variability in episodic memory

Learning and remembering are fundamental to our lives, so what causes us to forget? Answers often highlight preparatory processes that precede learning, as well as mnemonic processes during the act of encoding or retrieval. Importantly, evidence now indicates that preparatory processes that precede retrieval attempts also have powerful influences on memory success or failure. Here, we review recent work from neuroimaging, electroencephalography, pupillometry, and behavioral science to propose an integrative framework of retrieval-period dynamics that explains variance in remembering in the moment and across individuals as a function of interactions among preparatory attention, goal coding, and mnemonic processes. Extending this approach, we consider how a 'readiness to remember' (R2R) framework explains variance in high-level functions of memory and mnemonic disruptions in aging.

Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making

While often seeming to investigate rather different problems, research into value-based decision making and cognitive control have historically offered parallel insights into how people select thoughts and actions. While the former studies how people weigh costs and benefits to make a decision, the latter studies how they adjust information processing to achieve their goals. Recent work has highlighted ways in which decision-making research can inform our understanding of cognitive control. Here, we provide the complementary perspective: how cognitive control research has informed understanding of decision-making. We highlight three particular areas of research where this critical interchange has occurred: (1) how different types of goals shape the evaluation of choice options, (2) how people use control to adjust the ways they make their decisions, and (3) how people monitor decisions to inform adjustments to control at multiple levels and timescales. We show how adopting this alternate viewpoint offers new insight into the determinants of both decisions and control; provides alternative interpretations for common neuroeconomic findings; and generates fruitful directions for future research.

The whole prefrontal cortex is premotor cortex

We propose that the entirety of the prefrontal cortex (PFC) can be seen as fundamentally premotor in nature. By this, we mean that the PFC consists of an action abstraction hierarchy whose core function is the potentiation and depotentiation of possible action plans at different levels of granularity. We argue that the apex of the hierarchy should revolve around the process of goal-selection, which we posit is inherently a form of optimization over action abstraction. Anatomical and functional evidence supports the idea that this hierarchy originates on the orbital surface of the brain and extends dorsally to motor cortex. Accordingly, our viewpoint positions the orbitofrontal cortex in a key role in the optimization of goal-selection policies, and suggests that its other proposed roles are aspects of this more general function. Our proposed perspective will reframe outstanding questions, open up new areas of inquiry and align theories of prefrontal function with evolutionary principles. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

Value signals guide abstraction during learning

The human brain excels at constructing and using abstractions, such as rules, or concepts. Here, in two fMRI experiments, we demonstrate a mechanism of abstraction built upon the valuation of sensory features. Human volunteers learned novel association rules based on simple visual features. Reinforcement-learning algorithms revealed that, with learning, high-value abstract representations increasingly guided participant behaviour, resulting in better choices and higher subjective confidence. We also found that the brain area computing value signals – the ventromedial prefrontal cortex – prioritised and selected latent task elements during abstraction, both locally and through its connection to the visual cortex. Such a coding scheme predicts a causal role for valuation. Hence, in a second experiment, we used multivoxel neural reinforcement to test for the causality of feature valuation in the sensory cortex, as a mechanism of abstraction. Tagging the neural representation of a task feature with rewards evoked abstraction-based decisions. Together, these findings provide a novel interpretation of value as a goal-dependent, key factor in forging abstract representations.