The dimensionality of neural representations for control

Beyond Routine Maintenance: Current Trends in Working Memory Research

Working memory (WM) is an evolving concept. Our understanding of the neural functions that support WM develops iteratively alongside the approaches used to study it, and both can be profoundly shaped by available tools and prevailing theoretical paradigms. Here, the organizers of the 2024 Working Memory Symposium-inspired by this year's meeting-highlight current trends and looming questions in WM research. This review is organized into sections describing (1) ongoing efforts to characterize WM function across sensory modalities, (2) the growing appreciation that WM representations are malleable to context and future actions, (3) the enduring problem of how multiple WM items and features are structured and integrated, and (4) new insights about whether WM shares function with other cognitive processes that have conventionally been considered distinct. This review aims to chronicle where the field is headed and calls attention to issues that are paramount for future research.

Spatial and feature-selective attention interact to drive selective coding in frontoparietal cortex

Attention enables the selective processing of relevant information. Two types of selective attention, spatial and feature-selective attention, have separable neural effects but in real life are often used together. Here, we asked how these types of attention interact to affect information coding in a frontoparietal 'multiple-demand' (MD) network, essential for attentional control. Using functional magnetic resonance imaging (fMRI) with multivariate pattern analysis, we examined how covert attention to object features (colour or shape) and spatial locations (left or right) influences coding of task-related stimulus information. We found that spatial and feature-selective attention interacted multiplicatively on information coding in MD and visual regions, such that there was above-chance decoding of the attended feature of the attended object and no detectable coding of visually equivalent but behaviourally irrelevant aspects of the visual display. The attended information had a multidimensional neural representation, with stimulus information (e.g., colour) and discrimination difficulty (distance from the categorical decision boundary) reflected in separate dimensions. Rather than boosting processing of whole objects or relevant features across space, our results suggest neural activity reflects precise tuning to relevant information, indicating a highly selective control process that codes behaviourally relevant information across multiple dimensions.

Distributed Representations for Cognitive Control in Frontal Medial Cortex

In natural and artificial neural networks, modularity and distributed structure afford complementary but competing benefits. The former allows for hierarchical representations that can flexibly recombine modules to address novel problems, whereas the latter can benefit from less constrained training, potentially uncovering fruitful statistical regularities. Here, we investigate these competing demands in the context of human sequential behavior. First, we explore this setting by comparing the properties of several recurrent neural network models. We find that explicit hierarchical structure by itself fails to provide a critical performance advantage when compared with a "flat" model that does not incorporate hierarchical structure. However, hierarchy appears to facilitate cognitive control processes that support nonroutine behaviors and behaviors that are carried out under computational stress. Second, we compare these models against fMRI data using representational similarity analysis. We find that a model that incorporates so-called wiring costs in the cost function, which produces a hierarchically organized gradient of representational structure across the hidden layer of the neural network, best accounts for fMRI data collected from human participants in a previous study [Holroyd, C. B., Ribas-Fernandes, J. J. F., Shahnazian, D., Silvetti, M., & Verguts, T., Human midcingulate cortex encodes distributed representations of task progress. Proceedings of the National Academy of Sciences, U.S.A., 115, 6398-6403, 2018]. The results reveal that the ACC encodes distributed representations of sequential task context along a rostro-caudal gradient of abstraction: Rostral ACC encodes relatively abstract and temporally extended patterns of activity compared with those encoded by caudal ACC. These results provide insight into the role of ACC in motivation and cognitive control.

Stimulus representation in human frontal cortex supports flexible control in working memory

When holding visual information temporarily in working memory (WM), the neural representation of the memorandum is distributed across various cortical regions, including visual and frontal cortices. However, the role of stimulus representation in visual and frontal cortices during WM has been controversial. Here, we tested the hypothesis that stimulus representation persists in the frontal cortex to facilitate flexible control demands in WM. During functional MRI, participants flexibly switched between simple WM maintenance of visual stimulus or more complex rule-based categorization of maintained stimulus on a trial-by-trial basis. Our results demonstrated enhanced stimulus representation in the frontal cortex that tracked demands for active WM control and enhanced stimulus representation in the visual cortex that tracked demands for precise WM maintenance. This differential frontal stimulus representation traded off with the newly-generated category representation with varying control demands. Simulation using multi-module recurrent neural networks replicated human neural patterns when stimulus information was preserved for network readout. Altogether, these findings help reconcile the long-standing debate in WM research, and provide empirical and computational evidence that flexible stimulus representation in the frontal cortex during WM serves as a potential neural coding scheme to accommodate the ever-changing environment.

Novel Verbal Instructions Recruit Abstract Neural Patterns of Time-Variable Information Dimensionality

Human performance is endowed by neural representations of information that is relevant for behavior, some of which are also activated in a preparatory fashion to optimize later execution. Most studies to date have focused on highly practiced actions, leaving largely unaddressed the novel reconfiguration of information to generate unique whole task sets. Using electroencephalography, this study investigated the dynamics of the content and geometry reflected on the neural patterns of control representations during reconfiguration of information. We designed a verbal instruction paradigm where each trial involved novel combinations of multicomponent task information. By manipulating three task-relevant factors in a sample of 40 participants (26 females, 14 males), we observed complex coding schemes throughout the trial, during both preparation and implementation stages. The temporal profiles were consistent with a hierarchical structure: whereas task information was active in a sustained manner, the coding of more concrete stimulus features was more transient. Data showed both high dimensionality and abstraction, particularly during instruction encoding and target processing. Our results suggest that whenever task content could be recovered from neural patterns of activity, there was evidence of abstract coding, with an underlying geometry that favored generalization. During target processing, where potential interference across stimulus and response factors increased, orthogonal configurations also appeared. Overall, our findings uncover the dynamic manner with which control representations operate during novel recombination unique scenarios, with changes in dimensionality and abstraction adjusting along processing stages.

Geometry Linked to Untangling Efficiency Reveals Structure and Computation in Neural Populations

From an eagle spotting a fish in shimmering water to a scientist extracting patterns from noisy data, many cognitive tasks require untangling overlapping signals. Neural circuits achieve this by transforming complex sensory inputs into distinct, separable representations that guide behavior. Data-visualization techniques convey the geometry of these transformations, and decoding approaches quantify performance efficiency. However, we lack a framework for linking these two key aspects. Here we address this gap by introducing a data-driven analysis framework, which we call Geometry Linked to Untangling Efficiency (GLUE) with manifold capacity theory, that links changes in the geometrical properties of neural activity patterns to representational untangling at the computational level. We applied GLUE to over seven neuroscience datasets-spanning multiple organisms, tasks, and recording techniques-and found that task-relevant representations untangle in many domains, including along the cortical hierarchy, through learning, and over the course of intrinsic neural dynamics. Furthermore, GLUE can characterize the underlying geometric mechanisms of representational untangling, and explain how it facilitates efficient and robust computation. Beyond neuroscience, GLUE provides a powerful framework for quantifying information organization in data-intensive fields such as structural genomics and interpretable AI, where analyzing high-dimensional representations remains a fundamental challenge.

Ultrafast fMRI reveals serial queuing of information processing during multitasking in the human brain

The human brain is heralded for its massive parallel processing capacity, yet influential cognitive models suggest that there is a central bottleneck of information processing distinct from perceptual and motor stages that limits our ability to carry out two cognitively demanding tasks at once, resulting in the serial queuing of task information processing. Here we used ultrafast (199 ms TR), high-field (7T) fMRI with multivariate analyses to distinguish brain activity between two arbitrary sensorimotor response selection tasks when the tasks were temporally overlapping. We observed serial processing of task-specific activity in the fronto-parietal multiple-demand (MD) network, while processing in earlier sensory stages unfolded largely in parallel. Moreover, the MD network combined with modality-specific motor areas to define the functional characteristic of the central bottleneck at the stage of response selection. These results provide direct neural evidence for serial queuing of information processing and pinpoint the neural substrates undergirding the central bottleneck.

Lateral frontopolar theta-band activity supports flexible switching between emotion regulation strategies

Flexible emotion regulation is essential for mental health and well-being. However, neurocognitive mechanisms supporting emotion regulation flexibility remain unclear. Lateral frontal pole (FPl) contributes to flexible behavior by monitoring the efficacy of alternative strategies. This preregistered study examines if FPl also supports flexible use of emotion regulation strategies. It focuses on pre-decision theta-band activity as a potential indicator of this adaptive process. Sixty-three participants performed an emotion regulation strategy-switching task, involving three phases: (1) implementing an instructed (reappraisal or distraction) strategy, (2) deciding whether to 'maintain' the current or 'switch' to the alternative strategy, and (3) implementing the chosen strategy. Results showed that switching is predicted by the reduced efficacy of an initial emotion regulation strategy (indexed with EMG corrugator activity) and is made in accordance with situational demands (stimulus reappraisal affordances). Critically, switching to an alternative emotion regulation strategy is associated with increased theta-band power in FPl around the time of the decision. These findings support the previously established role of FPl theta-band activity in monitoring counterfactual efficacy of alternative strategies. Crucially, they extend this notion to cognitive emotion regulation, thereby offering promising neural targets for stimulation-based therapies aimed at enhancing emotion regulation flexibility in affective psychopathologies.

Dynamically shifting from compositional to conjunctive brain representations supports cognitive task learning

During cognitive task learning, neural representations must be rapidly constructed for novel task performance, then optimized for robust practiced task performance. How the geometry of neural representations changes to enable this transition from novel to practiced performance remains unknown. We hypothesized that practice involves a shift from compositional representations (task-general activity patterns that can be flexibly reused across tasks) to conjunctive representations (task-specific activity patterns specialized for the current task). Functional MRI during learning of multiple complex tasks substantiated this dynamic shift from compositional to conjunctive representations, which was associated with reduced cross-task interference (via pattern separation) and behavioral improvement. Further, we found that conjunctions originated in subcortex (hippocampus and cerebellum) and slowly spread to cortex, extending multiple memory systems theories to encompass cognitive task learning. The strengthening of conjunctive representations hence serves as a computational signature of learning, reflecting cortical-subcortical dynamics that optimize task representations in the human brain.

Highlights

Learning shifts multi-task representations from compositional to conjunctive formatsCortical conjunctions uniquely associate with improved behavior and pattern separationThese conjunctions strengthen over separated learning events and index switch costsSubcortical regions are critical for cross-region binding of task rule information.

Task structure tailors the geometry of neural representations in human lateral prefrontal cortex

How do human brains represent tasks of varying structure? The lateral prefrontal cortex (lPFC) flexibly represents task information. However, principles that shape lPFC representational geometry remain unsettled. We use fMRI and pattern analyses to reveal the structure of lPFC representational geometries as humans perform two distinct categorization tasks- one with flat, conjunctive categories and another with hierarchical, context-dependent categories. We show that lPFC encodes task-relevant information with task-tailored geometries of intermediate dimensionality. These geometries preferentially enhance the separability of task-relevant variables while encoding a subset in abstract form. Specifically, in the flat task, a global axis encodes response-relevant categories abstractly, while category-specific local geometries are high-dimensional. In the hierarchy task, a global axis abstractly encodes the higher-level context, while low-dimensional, context-specific local geometries compress irrelevant information and abstractly encode the relevant information. Comparing these task geometries exposes generalizable principles by which lPFC tailors representations to different tasks.

Distractor-specific control adaptation in multidimensional environments

Goal-directed behaviour requires humans to constantly manage and switch between multiple, independent and conflicting sources of information. Conventional cognitive control tasks, however, only feature one task and one source of distraction. Therefore, it is unclear how control is allocated in multidimensional environments. To address this question, we developed a multidimensional task-set interference paradigm, in which people need to manage distraction from three independent dimensions. We use this task to test whether people adapt to previous conflict by enhancing task-relevant information or suppressing task-irrelevant information. Three experiments provided strong evidence for the latter hypothesis. Moreover, control adaptation was highly dimension specific. Conflict from a given dimension only affected processing of that same dimension on subsequent trials, with no evidence for generalization. A new neural network model shows that our results can only be simulated when including multiple independent conflict-detector units. Our results call for an update to classic models of cognitive control and their neurocomputational underpinnings.

Understanding cognitive processes across spatial scales of the brain

Cognition arises from neural operations at multiple spatial scales, from individual neurons to large-scale networks. Despite extensive research on coding principles and emergent cognitive processes across brain areas, investigation across scales has been limited. Here, we propose ways to test the idea that different cognitive processes emerge from distinct information coding principles at various scales, which collectively give rise to complex behavior. This approach involves comparing brain-behavior associations and the underlying neural geometry across scales, alongside an investigation of global and local scale interactions. Bridging findings across species and techniques through open science and collaborations is essential to comprehensively understand the multiscale brain and its functions.

Compositional architecture: Orthogonal neural codes for task context and spatial memory in prefrontal cortex

The prefrontal cortex (PFC) is crucial for maintaining working memory across diverse cognitive tasks, yet how it adapts to varying task demands remains unclear. Compositional theories propose that cognitive processes in neural network rely on shared components that can be reused to support different behaviors. However, previous studies have suggested that working memory components are task specific, challenging this framework. Here, we revisit this question using a population-based approach. We recorded neural activity in macaque monkeys performing two spatial working memory tasks with opposing goals: one requiring movement toward previously presented spatial locations (look task) and the other requiring avoidance of those locations (no-look task). Despite differences in task demands, we found that spatial memory representations were largely conserved at the population level, with a common low-dimensional neural subspace encoding memory across both tasks. In parallel, task identity was encoded in an orthogonal subspace, providing a stable and independent representation of contextual information. These results provide neural evidence for a compositional model of working memory, where representational geometry enables the efficient and flexible reuse of mnemonic codes across behavioral contexts while maintaining an independent representation of context.

Exploring the role of dimensionality transformation in episodic memory

Episodic memory must accomplish two adversarial goals: encoding and storing a multitude of experiences without exceeding the finite neuronal structure of the brain, and recalling memories in vivid detail. Dimensionality reduction and expansion ('dimensionality transformation') enable the brain to meet these demands. Reduction compresses sensory input into simplified, storable codes, while expansion reconstructs vivid details. Although these processes are essential to memory, their neural mechanisms for episodic memory remain unclear. Drawing on recent insights from cognitive psychology, systems neuroscience, and neuroanatomy, we propose two accounts of how dimensionality transformation occurs in the brain: structurally (via corticohippocampal pathways) and functionally (through neural oscillations). By examining cross-species evidence, we highlight neural mechanisms that may support episodic memory and identify crucial questions for future research.

The Dimensionality of Neural Coding for Cognitive Control Is Gradually Transformed within the Lateral Prefrontal Cortex

Cognitive control relies on neural representations that are inherently high-dimensional and distributed across multiple subregions in the prefrontal cortex (PFC). Traditional approaches tackle prefrontal representation by reducing it into a unidimensional measure (univariate amplitude) or using it to distinguish a limited number of alternatives (pattern classification). In contrast, representational similarity analysis (RSA) enables flexibly formulating various hypotheses about informational contents underlying the neural codes, explicitly comparing hypotheses, and examining the representational alignment between brain regions. Here, we used a multifaceted paradigm wherein the difficulty of cognitive control was manipulated separately for five cognitive tasks. We used RSA to unveil representational contents, measure the representational alignment between regions, and quantify representational generality versus specificity. We found a graded transition in the lateral PFC: The dorsocaudal PFC was tuned to task difficulty (indexed by reaction times), preferentially connected with the parietal cortex, and representationally generalizable across domains. The ventrorostral PFC was tuned to the abstract structure of tasks, preferentially connected with the temporal cortex, and representationally specific. The middle PFC (interposed between the dorsocaudal and ventrorostral PFC) was tuned to individual task sets and ranked in the middle in terms of connectivity and generalizability. Furthermore, whether a region was dimensionally rich or sparse covaried with its functional profile: Low dimensionality (only gist) in the dorsocaudal PFC dovetailed with better generality, whereas high dimensionality (gist plus details) in the ventrorostral PFC corresponded with better ability to encode subtleties. Our findings, collectively, demonstrate how cognitive control is decomposed into distinct facets that transition steadily along prefrontal subregions.

Theory of morphodynamic information processing: Linking sensing to behaviour

The traditional understanding of brain function has predominantly focused on chemical and electrical processes. However, new research in fruit fly (Drosophila) binocular vision reveals ultrafast photomechanical photoreceptor movements significantly enhance information processing, thereby impacting a fly's perception of its environment and behaviour. The coding advantages resulting from these mechanical processes suggest that similar physical motion-based coding strategies may affect neural communication ubiquitously. The theory of neural morphodynamics proposes that rapid biomechanical movements and microstructural changes at the level of neurons and synapses enhance the speed and efficiency of sensory information processing, intrinsic thoughts, and actions by regulating neural information in a phasic manner. We propose that morphodynamic information processing evolved to drive predictive coding, synchronising cognitive processes across neural networks to match the behavioural demands at hand effectively.

Multitasking Practice Eliminates Modality-Based Interference by Separating Task Representations in Sensory Brain Regions

The debate on the neural basis of multitasking costs evolves around neural overlap between concurrently performed tasks. Recent evidence suggests that training-related reductions in representational overlap in fronto-parietal brain regions predict multitasking improvements. Cognitive theories assume that overlap of task representations may lead to unintended information exchange between tasks (i.e., crosstalk). Modality-based crosstalk was suggested as a source for multitasking costs in multisensory settings. Robust findings of increased costs for certain modality mappings may be explained by crosstalk between the stimulus modality in one task and sensory action consequences in the concurrently performed task. Whether modality-based crosstalk emerges from representational overlap in general fronto-parietal multitasking regions or modality-specific regions is not known yet. In this functional neuroimaging study, we investigate neural overlap during multitasking performance in humans, focusing on modality compatibility by employing multivariate pattern analysis and modality-specific practice interventions in three groups (total N = 54, 24 females). We observed significant differences between modality compatible and modality incompatible single-task representations, specifically in the auditory cortex but not in fronto-parietal regions. Notably, improved auditory decoding accuracy related to modality incompatible tasks was predictive of performance gains in the corresponding dual task along with complete elimination of modality-specific dual-task costs. This predictive relationship was evident only in the group practicing modality incompatible mappings, suggesting that specific practice on task sets with modality overlap influenced both neural representations and subsequent multitasking performance. This study contributes to the integration of cognitive theory and neuroscience and the role of task representations in dual-task interference.

Cognitive Control

Humans and other primates have a remarkable ability to perform a wide range of tasks and behaviors, even novel ones, in order to achieve their goals. Further, they are able to shift flexibly among these behaviors as the contexts demand. Cognitive control is the function at the base of this remarkable behavioral generativity and flexibility. The present review provides a survey of current research on cognitive control focusing on two of its primary features within a control systems framework: (a) the ability to select new behaviors based on context and (b) the ability to monitor ongoing behavior and adjust accordingly. Throughout, the review places an emphasis on how differences in the content and structure of task representations affect these core features of cognitive control.

Neuronal encoding of behaviors and instrumental learning in the dorsal striatum

The dorsal striatum is instrumental in regulating motor control and goal-directed behaviors. The classical description of the two output pathways of the dorsal striatum highlights their antagonistic control over actions. However, recent experimental evidence implicates both pathways and their coordinated activities during actions. In this review, we examine the different models proposed for striatal encoding of actions during self-paced behaviors and how they can account for evidence harvested during goal-directed behaviors. We also discuss how the activation of striatal ensembles can be reshaped and reorganized to support the formation of instrumental learning and behavioral flexibility. Future work integrating these considerations may resolve controversies regarding the control of actions by striatal networks.

The representation of abstract goals in working memory is supported by task-congruent neural geometry

Successful goal-directed behavior requires the maintenance and implementation of abstract task goals on concrete stimulus information in working memory. Previous working memory research has revealed distributed neural representations of task information across cortex. However, how the distributed task representations emerge and communicate with stimulus-specific information to implement flexible goal-directed computations is still unclear. Here, leveraging electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) in human participants along with state space analyses, we provided converging evidence in support of a low-dimensional neural geometry of goal information congruent with a designed task space, which first emerged in frontal cortex during goal maintenance and then transferred to posterior cortex through frontomedial-to-posterior theta coherence for implementation on stimulus-specific representations. Importantly, the fidelity of the goal geometry was associated with memory performance. Collectively, our findings suggest that abstract goals in working memory are represented in an organized, task-congruent neural geometry for communications from frontal to posterior cortex to enable computations necessary for goal-directed behaviors.

The Dimensions of dimensionality

Cognitive scientists often infer multidimensional representations from data. Whether the data involve text, neuroimaging, neural networks, or human judgments, researchers frequently infer and analyze latent representational spaces (i.e., embeddings). However, the properties of a latent representation (e.g., prediction performance, interpretability, compactness) depend on the inference procedure, which can vary widely across endeavors. For example, dimensions are not always globally interpretable and the dimensionality of different embeddings may not be readily comparable. Moreover, the dichotomy between multidimensional spaces and purportedly richer representational formats, such as graph representations, is misleading. We review what the different notions of dimension in cognitive science imply for how these latent representations should be used and interpreted.

Recurrent Connectivity Shapes Spatial Coding in Hippocampal CA3 Subregions

Stable and flexible neural representations of space in the hippocampus are crucial for navigating complex environments. However, how these distinct representations emerge from the underlying local circuit architecture remains unknown. Using two-photon imaging of CA3 subareas during active behavior, we reveal opposing coding strategies within specific CA3 subregions, with proximal neurons demonstrating stable and generalized representations and distal neurons showing dynamic and context-specific activity. We show in artificial neural network models that varying the recurrence level causes these differences in coding properties to emerge. We confirmed the contribution of recurrent connectivity to functional heterogeneity by characterizing the representational geometry of neural recordings and comparing it with theoretical predictions of neural manifold dimensionality. Our results indicate that local circuit organization, particularly recurrent connectivity among excitatory neurons, plays a key role in shaping complementary spatial representations within the hippocampus.

Language proficiency is associated with neural representational dimensionality of semantic concepts

Previous studies suggest that semantic concepts are characterized by high-dimensional neural representations and that language proficiency affects semantic processing. However, it is not clear whether language proficiency modulates the dimensional representations of semantic concepts at the neural level. To address this question, the present study adopted principal component analysis (PCA) and representational similarity analysis (RSA) to examine the differences in representational dimensionalities (RDs) and in semantic representations between words in highly proficient (Chinese) and less proficient (English) language. PCA results revealed that language proficiency increased the dimensions of lexical representations in the left inferior frontal gyrus, temporal pole, inferior temporal gyrus, supramarginal gyrus, angular gyrus, and fusiform gyrus. RSA results further showed that these regions represented semantic information and that higher semantic representations were observed in highly proficient language relative to less proficient language. These results suggest that language proficiency is associated with the neural representational dimensionality of semantic concepts.

Flexible control of sequence working memory in the macaque frontal cortex

To memorize a sequence, one must serially bind each item to its rank order. How the brain controls a given input to bind its associated order in sequence working memory (SWM) remains unexplored. Here, we investigated the neural representations underlying SWM control using electrophysiological recordings in the frontal cortex of macaque monkeys performing forward and backward SWM tasks. Separate and generalizable low-dimensional subspaces for sensory and memory information were found within the same frontal circuitry, and SWM control was reflected in these neural subspaces' organized dynamics. Each item at each rank was sequentially entered into a common sensory subspace and, depending on forward or backward task requirement, flexibly and timely sent into rank-selective SWM subspaces. Neural activity in these SWM subspaces faithfully predicted the recalled item and order information in single error trials. Thus, compositional neural population codes with well-orchestrated dynamics in frontal cortex support the flexible control of SWM.

Distinct functions for beta and alpha bursts in gating of human working memory

Multiple neural mechanisms underlying gating to working memory have been proposed with divergent results obtained in human and animal studies. Previous findings from non-human primates suggest prefrontal beta frequency bursts as a correlate of transient inhibition during selective encoding. Human studies instead suggest a similar role for sensory alpha power fluctuations. To cast light on these discrepancies we employed a sequential working memory task with distractors for human participants. In particular, we examined their whole-brain electrophysiological activity in both alpha and beta bands with the same single-trial burst analysis earlier performed on non-human primates. Our results reconcile earlier findings by demonstrating that both alpha and beta bursts in humans correlate with the filtering and control of memory items, but with region and task-specific differences between the two rhythms. Occipital beta burst patterns were selectively modulated during the transition from sensory processing to memory retention whereas prefrontal and parietal beta bursts tracked sequence order and were proactively upregulated prior to upcoming target encoding. Occipital alpha bursts instead increased during the actual presentation of unwanted sensory stimuli. Source reconstruction additionally suggested the involvement of striatal and thalamic alpha and beta. Thus, specific whole-brain burst patterns correlate with different aspects of working memory control.

A transient high-dimensional geometry affords stable conjunctive subspaces for efficient action selection

Flexible action selection requires cognitive control mechanisms capable of mapping the same inputs to different output actions depending on the context. From a neural state-space perspective, this requires a control representation that separates similar input neural states by context. Additionally, for action selection to be robust and time-invariant, information must be stable in time, enabling efficient readout. Here, using EEG decoding methods, we investigate how the geometry and dynamics of control representations constrain flexible action selection in the human brain. Participants performed a context-dependent action selection task. A forced response procedure probed action selection different states in neural trajectories. The result shows that before successful responses, there is a transient expansion of representational dimensionality that separated conjunctive subspaces. Further, the dynamics stabilizes in the same time window, with entry into this stable, high-dimensional state predictive of individual trial performance. These results establish the neural geometry and dynamics the human brain needs for flexible control over behavior.

Practice Reshapes the Geometry and Dynamics of Task-tailored Representations

Extensive practice makes task performance more efficient and precise, leading to automaticity. However, theories of automaticity differ on which levels of task representations (e.g., low-level features, stimulus-response mappings, or high-level conjunctive memories of individual events) change with practice, despite predicting the same pattern of improvement (e.g., power law of practice). To resolve this controversy, we built on recent theoretical advances in understanding computations through neural population dynamics. Specifically, we hypothesized that practice optimizes the neural representational geometry of task representations to minimally separate the highest-level task contingencies needed for successful performance. This involves efficiently reaching conjunctive neural states that integrate task-critical features nonlinearly while abstracting over non-critical dimensions. To test this hypothesis, human participants (n = 40) engaged in extensive practice of a simple, context-dependent action selection task over 3 days while recording EEG. During initial rapid improvement in task performance, representations of the highest-level, context-specific conjunctions of task-features were enhanced as a function of the number of successful episodes. Crucially, only enhancement of these conjunctive representations, and not lower-order representations, predicted the power-law improvement in performance. Simultaneously, over sessions, these conjunctive neural states became more stable earlier in time and more aligned, abstracting over redundant task features, which correlated with offline performance gain in reducing switch costs. Thus, practice optimizes the dynamic representational geometry as task-tailored neural states that minimally tesselate the task space, taming their high-dimensionality.

A Transient High-dimensional Geometry Affords Stable Conjunctive Subspaces for Efficient Action Selection

Flexible action selection requires cognitive control mechanisms capable of mapping the same inputs to different output actions depending on the context. From a neural state-space perspective, this requires a control representation that separates similar input neural states by context. Additionally, for action selection to be robust and time-invariant, information must be stable in time, enabling efficient readout. Here, using EEG decoding methods, we investigate how the geometry and dynamics of control representations constrain flexible action selection in the human brain. Participants performed a context-dependent action selection task. A forced response procedure probed action selection different states in neural trajectories. The result shows that before successful responses, there is a transient expansion of representational dimensionality that separated conjunctive subspaces. Further, the dynamics stabilizes in the same time window, with entry into this stable, high-dimensional state predictive of individual trial performance. These results establish the neural geometry and dynamics the human brain needs for flexible control over behavior.

Thalamocortical architectures for flexible cognition and efficient learning

The brain exhibits a remarkable ability to learn and execute context-appropriate behaviors. How it achieves such flexibility, without sacrificing learning efficiency, is an important open question. Neuroscience, psychology, and engineering suggest that reusing and repurposing computations are part of the answer. Here, we review evidence that thalamocortical architectures may have evolved to facilitate these objectives of flexibility and efficiency by coordinating distributed computations. Recent work suggests that distributed prefrontal cortical networks compute with flexible codes, and that the mediodorsal thalamus provides regularization to promote efficient reuse. Thalamocortical interactions resemble hierarchical Bayesian computations, and their network implementation can be related to existing gating, synchronization, and hub theories of thalamic function. By reviewing recent findings and providing a novel synthesis, we highlight key research horizons integrating computation, cognition, and systems neuroscience.

Mixed selectivity: Cellular computations for complexity

The property of mixed selectivity has been discussed at a computational level and offers a strategy to maximize computational power by adding versatility to the functional role of each neuron. Here, we offer a biologically grounded implementational-level mechanistic explanation for mixed selectivity in neural circuits. We define pure, linear, and nonlinear mixed selectivity and discuss how these response properties can be obtained in simple neural circuits. Neurons that respond to multiple, statistically independent variables display mixed selectivity. If their activity can be expressed as a weighted sum, then they exhibit linear mixed selectivity; otherwise, they exhibit nonlinear mixed selectivity. Neural representations based on diverse nonlinear mixed selectivity are high dimensional; hence, they confer enormous flexibility to a simple downstream readout neural circuit. However, a simple neural circuit cannot possibly encode all possible mixtures of variables simultaneously, as this would require a combinatorially large number of mixed selectivity neurons. Gating mechanisms like oscillations and neuromodulation can solve this problem by dynamically selecting which variables are mixed and transmitted to the readout.

Orthogonality of sensory and contextual categorical dynamics embedded in a continuum of responses from the second somatosensory cortex

How does the brain simultaneously process signals that bring complementary information, like raw sensory signals and their transformed counterparts, without any disruptive interference? Contemporary research underscores the brain's adeptness in using decorrelated responses to reduce such interference. Both neurophysiological findings and artificial neural networks support the notion of orthogonal representation for signal differentiation and parallel processing. Yet, where, and how raw sensory signals are transformed into more abstract representations remains unclear. Using a temporal pattern discrimination task in trained monkeys, we revealed that the second somatosensory cortex (S2) efficiently segregates faithful and transformed neural responses into orthogonal subspaces. Importantly, S2 population encoding for transformed signals, but not for faithful ones, disappeared during a nondemanding version of this task, which suggests that signal transformation and their decoding from downstream areas are only active on-demand. A mechanistic computation model points to gain modulation as a possible biological mechanism for the observed context-dependent computation. Furthermore, individual neural activities that underlie the orthogonal population representations exhibited a continuum of responses, with no well-determined clusters. These findings advocate that the brain, while employing a continuum of heterogeneous neural responses, splits population signals into orthogonal subspaces in a context-dependent fashion to enhance robustness, performance, and improve coding efficiency.

Beta: bursts of cognition

Beta oscillations are linked to the control of goal-directed processing of sensory information and the timing of motor output. Recent evidence demonstrates they are not sustained but organized into intermittent high-power bursts mediating timely functional inhibition. This implies there is a considerable moment-to-moment variation in the neural dynamics supporting cognition. Beta bursts thus offer new opportunities for studying how sensory inputs are selectively processed, reshaped by inhibitory cognitive operations and ultimately result in motor actions. Recent method advances reveal diversity in beta bursts that provide deeper insights into their function and the underlying neural circuit activity motifs. We propose that brain-wide, spatiotemporal patterns of beta bursting reflect various cognitive operations and that their dynamics reveal nonlinear aspects of cortical processing.

Cognition is an emergent property

Cognition relies on the flexible organization of neural activity. In this discussion, we explore how many aspects of this organization can be described as emergent properties, not reducible to their constituent parts. We discuss how electrical fields in the brain can serve as a medium for propagating activity nearly instantaneously, and how population-level patterns of neural activity can organize computations through subspace coding.

Nonfrontal Control of Working Memory

Items held in visual working memory can be quickly updated, replaced, removed, and even manipulated in accordance with current behavioral goals. Here, we use multivariate pattern analyses to identify the patterns of neuronal activity that realize the executive control processes supervising these flexible stores. We find that portions of the middle temporal gyrus and the intraparietal sulcus represent what item is cued for continued memorization independently of representations of the item itself. Importantly, this selection-specific activity could not be explained by sensory representations of the cue and is only present when control is exerted. Our results suggest that the selection of memorized items might be controlled in a distributed and decentralized fashion. This evidence provides an alternative perspective to the notion of "domain general" central executive control over memory function.

Orthogonal neural encoding of targets and distractors supports multivariate cognitive control

The complex challenges of our mental life require us to coordinate multiple forms of neural information processing. Recent behavioural studies have found that people can coordinate multiple forms of attention, but the underlying neural control process remains obscure. We hypothesized that the brain implements multivariate control by independently monitoring feature-specific difficulty and independently prioritizing feature-specific processing. During functional MRI, participants performed a parametric conflict task that separately tags target and distractor processing. Consistent with feature-specific monitoring, univariate analyses revealed spatially segregated encoding of target and distractor difficulty in the dorsal anterior cingulate cortex. Consistent with feature-specific attentional priority, our encoding geometry analysis revealed overlapping but orthogonal representations of target and distractor coherence in the intraparietal sulcus. Coherence representations were mediated by control demands and aligned with both performance and frontoparietal activity, consistent with top-down attention. Together, these findings provide evidence for the neural geometry necessary to coordinate multivariate cognitive control.

Spontaneously emerging internal models of visual sequences combine abstract and event-specific information in the prefrontal cortex

When exposed to sensory sequences, do macaque monkeys spontaneously form abstract internal models that generalize to novel experiences? Here, we show that neuronal populations in macaque ventrolateral prefrontal cortex jointly encode visual sequences by separate codes for the specific pictures presented and for their abstract sequential structure. We recorded prefrontal neurons while macaque monkeys passively viewed visual sequences and sequence mismatches in the local-global paradigm. Even without any overt task or response requirements, prefrontal populations spontaneously form representations of sequence structure, serial order, and image identity within distinct but superimposed neuronal subspaces. Representations of sequence structure rapidly update following single exposure to a mismatch sequence, while distinct populations represent mismatches for sequences of different complexity. Finally, those representations generalize across sequences following the same repetition structure but comprising different images. These results suggest that prefrontal populations spontaneously encode rich internal models of visual sequences reflecting both content-specific and abstract information.

The mediodorsal thalamus in executive control

Executive control, the ability to organize thoughts and action plans in real time, is a defining feature of higher cognition. Classical theories have emphasized cortical contributions to this process, but recent studies have reinvigorated interest in the role of the thalamus. Although it is well established that local thalamic damage diminishes cognitive capacity, such observations have been difficult to inform functional models. Recent progress in experimental techniques is beginning to enrich our understanding of the anatomical, physiological, and computational substrates underlying thalamic engagement in executive control. In this review, we discuss this progress and particularly focus on the mediodorsal thalamus, which regulates the activity within and across frontal cortical areas. We end with a synthesis that highlights frontal thalamocortical interactions in cognitive computations and discusses its functional implications in normal and pathological conditions.

Domain general frontoparietal regions show modality-dependent coding of auditory and visual rules

A defining feature of human cognition is our ability to respond flexibly to what we see and hear, changing how we respond depending on our current goals. In fact, we can rapidly associate almost any input stimulus with any arbitrary behavioural response. This remarkable ability is thought to depend on a frontoparietal "multiple demand" circuit which is engaged by many types of cognitive demand and widely referred to as domain general. However, it is not clear how responses to multiple input modalities are structured within this system. Domain generality could be achieved by holding information in an abstract form that generalises over input modality, or in a modality-tagged form, which uses similar resources but produces unique codes to represent the information in each modality. We used a stimulus-response task, with conceptually identical rules in two sensory modalities (visual and auditory), to distinguish between these possibilities. Multivariate decoding of functional magnetic resonance imaging data showed that representations of visual and auditory rules recruited overlapping neural resources but were expressed in modality-tagged non-generalisable neural codes. Our data suggest that this frontoparietal system may draw on the same or similar resources to solve multiple tasks, but does not create modality-general representations of task rules, even when those rules are conceptually identical between domains.

Dorsolateral prefrontal activity supports a cognitive space organization of cognitive control

Cognitive control resolves conflicts between task-relevant and -irrelevant information to enable goal-directed behavior. As conflicts can arise from different sources (e.g., sensory input, internal representations), how a limited set of cognitive control processes can effectively address diverse conflicts remains a major challenge. Based on the cognitive space theory, different conflicts can be parameterized and represented as distinct points in a (low-dimensional) cognitive space, which can then be resolved by a limited set of cognitive control processes working along the dimensions. It leads to a hypothesis that conflicts similar in their sources are also represented similarly in the cognitive space. We designed a task with five types of conflicts that could be conceptually parameterized. Both human performance and fMRI activity patterns in the right dorsolateral prefrontal cortex support that different types of conflicts are organized based on their similarity, thus suggesting cognitive space as a principle for representing conflicts.

Memory for abstract control states does not decay with increasing retrieval delays

Recent studies have suggested that abstract control states (i.e., internal attentional states independent from concrete stimuli and responses) can be stored in episodic memory and retrieved subsequently. However, the duration of such a control state memory remains unclear. Previous research has found a quick and complete decay for stimulus-response bindings after 2000-5000 ms. Here, we tested a possible decay of control state bindings with retrieval delays of 2000, 3000, or 5000 ms. Five preregistered experiments used a confound-minimized prime-target task to measure the congruency sequence effect (CSE) separately for trials in which a nominally irrelevant context feature changed or repeated across trials. Analyses of the individual experiments did not result in conclusive evidence. A mega-analysis integrating the data of all experiments (Ntotal = 326) replicated evidence for binding and retrieval of control states, in that larger CSEs were found for context repetition trials. Importantly, Bayesian analysis indicated that this effect was not modulated by the length of retrieval delay. While this finding suggests that bindings of abstract control states can be relatively robust, we also discuss possible limitations of the present research.

New organizational principles and 3D cytoarchitectonic maps of the dorsolateral prefrontal cortex in the human brain

Areas of the dorsolateral prefrontal cortex (DLPFC) are part of the frontoparietal control, default mode, salience, and ventral attention networks. The DLPFC is involved in executive functions, like working memory, value encoding, attention, decision-making, and behavioral control. This functional heterogeneity is not reflected in existing neuroanatomical maps. For example, previous cytoarchitectonic studies have divided the DLPFC into two or four areas. Macroanatomical parcellations of this region rely on gyri and sulci, which are not congruent with cytoarchitectonic parcellations. Therefore, this study aimed to provide a microstructural analysis of the human DLPFC and 3D maps of cytoarchitectonic areas to help address the observed functional variability in studies of the DLPFC. We analyzed ten human post-mortem brains in serial cell-body stained brain sections and mapped areal boundaries using a statistical image analysis approach. Five new areas (i.e., SFG2, SFG3, SFG4, MFG4, and MFG5) were identified on the superior and middle frontal gyrus, i.e., regions corresponding to parts of Brodmann areas 9 and 46. Gray level index profiles were used to determine interregional cytoarchitectural differences. The five new areas were reconstructed in 3D, and probability maps were generated in commonly used reference spaces, considering the variability of areas in stereotaxic space. Hierarchical cluster analysis revealed a high degree of similarity within the identified DLPFC areas while neighboring areas (frontal pole, Broca's region, area 8, and motoric areas) were separable. Comparisons with functional imaging studies revealed specific functional profiles of the DLPFC areas. Our results indicate that the new areas do not follow a simple organizational gradient assumption in the DLPFC. Instead, they are more similar to those of the ventrolateral prefrontal cortex (Broca's areas 44, 45) and frontopolar areas (Fp1, Fp2) than to the more posterior areas. Within the DLPFC, the cytoarchitectonic similarities between areas do not seem to follow a simple anterior-to-posterior gradient either, but cluster along other principles. The new maps are part of the publicly available Julich Brain Atlas and provide a microstructural reference for existing and future imaging studies. Thus, our study represents a further step toward deciphering the structural-functional organization of the human prefrontal cortex.

The emergence of task-relevant representations in a nonlinear decision-making task

This paper describes the relationship between performance in a decision-making task and the emergence of task-relevant representations. Participants learnt two tasks in which the appropriate response depended on multiple relevant stimuli and the underlying stimulus-outcome associations were governed by a latent feature that participants could discover. We divided participants into good and bad performers based on their overall classification rate and computed behavioural accuracy for each feature value. We found that participants with better performance had a better representation of the latent feature space. We then used representation similarity analysis on Electroencephalographic (EEG) data to identify when these representations emerge. We were able to decode task-relevant representations in a time window emerging 700 ms after stimulus presentation, but only for participants with good task performance. Our findings suggest that, in order to make good decisions, it is necessary to create and extract a low-dimensional representation of the task at hand.

Policy abstraction as a predictor of cognitive effort avoidance

Consistent evidence has established that people avoid cognitively effortful tasks. However, the features that make a task cognitively effortful are still not well understood. Multiple hypotheses have been proposed regarding which task demands underlie cognitive effort costs, such as time-on-task, error likelihood, and the general engagement of cognitive control. In this study, we test the novel hypothesis that tasks requiring behavior according to higher degrees of policy abstraction are experienced as more effortful. Accordingly, policy abstraction, operationalized as the levels of contextual contingency required by task rules, drives task avoidance over and above the effects of task performance, such as time-on-task or error likelihood. To test this hypothesis, we combined two previously established cognitive control tasks that parametrically manipulated policy abstraction with the demand selection task procedure. The design of these tasks allowed us to test whether people avoided tasks with higher order policy abstraction while controlling for the contribution of factors such as time-on-task and expected error rate (ER). Consistent with our hypothesis, we observed that policy abstraction was the strongest predictor of cognitive effort choices, followed by ER. This was evident across both studies and in a within-subject cross-study analysis. These results establish at least one task feature independent of performance, which is predictive of task avoidance behavior. We interpret these results within an opportunity cost framework for understanding aversive experiences of cognitive effort while performing a task. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Neural representations for multi-context visuomotor adaptation and the impact of common representation on multi-task performance: a multivariate decoding approach

The human brain's remarkable motor adaptability stems from the formation of context representations and the use of a common context representation (e.g., an invariant task structure across task contexts) derived from structural learning. However, direct evaluation of context representations and structural learning in sensorimotor tasks remains limited. This study aimed to rigorously distinguish neural representations of visual, movement, and context levels crucial for multi-context visuomotor adaptation and investigate the association between representation commonality across task contexts and adaptation performance using multivariate decoding analysis with fMRI data. Here, we focused on three distinct task contexts, two of which share a rotation structure (i.e., visuomotor rotation contexts with -90° and +90° rotations, in which the mouse cursor's movement was rotated 90 degrees counterclockwise and clockwise relative to the hand-movement direction, respectively) and the remaining one does not (i.e., mirror-reversal context where the horizontal movement of the computer mouse was inverted). This study found that visual representations (i.e., visual direction) were decoded in the occipital area, while movement representations (i.e., hand-movement direction) were decoded across various visuomotor-related regions. These findings are consistent with prior research and the widely recognized roles of those areas. Task-context representations (i.e., either -90° rotation, +90° rotation, or mirror-reversal) were also distinguishable in various brain regions. Notably, these regions largely overlapped with those encoding visual and movement representations. This overlap suggests a potential intricate dependency of encoding visual and movement directions on the context information. Moreover, we discovered that higher task performance is associated with task-context representation commonality, as evidenced by negative correlations between task performance and task-context-decoding accuracy in various brain regions, potentially supporting structural learning. Importantly, despite limited similarities between tasks (e.g., rotation and mirror-reversal contexts), such association was still observed, suggesting an efficient mechanism in the brain that extracts commonalities from different task contexts (such as visuomotor rotations or mirror-reversal) at multiple structural levels, from high-level abstractions to lower-level details. In summary, while illuminating the intricate interplay between visuomotor processing and context information, our study highlights the efficiency of learning mechanisms, thereby paving the way for future exploration of the brain's versatile motor ability.

Subspace partitioning in the human prefrontal cortex resolves cognitive interference

The human prefrontal cortex (PFC) constitutes the structural basis underlying flexible cognitive control, where mixed-selective neural populations encode multiple task features to guide subsequent behavior. The mechanisms by which the brain simultaneously encodes multiple task-relevant variables while minimizing interference from task-irrelevant features remain unknown. Leveraging intracranial recordings from the human PFC, we first demonstrate that competition between coexisting representations of past and present task variables incurs a behavioral switch cost. Our results reveal that this interference between past and present states in the PFC is resolved through coding partitioning into distinct low-dimensional neural states; thereby strongly attenuating behavioral switch costs. In sum, these findings uncover a fundamental coding mechanism that constitutes a central building block of flexible cognitive control.

Adaptive coding of stimulus information in human frontoparietal cortex during visual classification

The neural mechanisms of how frontal and parietal brain regions support flexible adaptation of behavior remain poorly understood. Here, we used functional magnetic resonance imaging (fMRI) and representational similarity analysis (RSA) to investigate frontoparietal representations of stimulus information during visual classification under varying task demands. Based on prior research, we predicted that increasing perceptual task difficulty should lead to adaptive changes in stimulus coding: task-relevant category information should be stronger, while task-irrelevant exemplar-level stimulus information should become weaker, reflecting a focus on the behaviorally relevant category information. Counter to our expectations, however, we found no evidence for adaptive changes in category coding. We did find weakened coding at the exemplar-level within categories however, demonstrating that task-irrelevant information is de-emphasized in frontoparietal cortex. These findings reveal adaptive coding of stimulus information at the exemplar-level, highlighting how frontoparietal regions might support behavior even under challenging conditions.

Uncertainty aversion predicts the neural expansion of semantic representations

Correctly identifying the meaning of a stimulus requires activating the appropriate semantic representation among many alternatives. One way to reduce this uncertainty is to differentiate semantic representations from each other, thereby expanding the semantic space. Here, in four experiments, we test this semantic-expansion hypothesis, finding that uncertainty-averse individuals exhibit increasingly differentiated and separated semantic representations. This effect is mirrored at the neural level, where uncertainty aversion predicts greater distances between activity patterns in the left inferior frontal gyrus when reading words, and enhanced sensitivity to the semantic ambiguity of these words in the ventromedial prefrontal cortex. Two direct tests of the behavioural consequences of semantic expansion further reveal that uncertainty-averse individuals exhibit reduced semantic interference and poorer generalization. Together, these findings show that the internal structure of our semantic representations acts as an organizing principle to make the world more identifiable.

Comparing representations and computations in single neurons versus neural networks

Single-neuron-level explanations have been the gold standard in neuroscience for decades. Recently, however, neural-network-level explanations have become increasingly popular. This increase in popularity is driven by the fact that the analysis of neural networks can solve problems that cannot be addressed by analyzing neurons independently. In this opinion article, I argue that while both frameworks employ the same general logic to link physical and mental phenomena, in many cases the neural network framework provides better explanatory objects to understand representations and computations related to mental phenomena. I discuss what constitutes a mechanistic explanation in neural systems, provide examples, and conclude by highlighting a number of the challenges and considerations associated with the use of analyses of neural networks to study brain function.

Geometric transformation of cognitive maps for generalization across hippocampal-prefrontal circuits

The ability to abstract information to guide decisions during navigation across changing environments is essential for adaptation and requires the integrity of the hippocampal-prefrontal circuitry. The hippocampus encodes navigational information in a cognitive map, but it remains unclear how cognitive maps are transformed across hippocampal-prefrontal circuits to support abstraction and generalization. Here, we simultaneously record hippocampal-prefrontal ensembles as rats generalize navigational rules across distinct environments. We find that, whereas hippocampal representational maps maintain specificity of separate environments, prefrontal maps generalize across environments. Furthermore, while both maps are structured within a neural manifold of population activity, they have distinct representational geometries. Prefrontal geometry enables abstraction of rule-informative variables, a representational format that generalizes to novel conditions of existing variable classes. Hippocampal geometry lacks such abstraction. Together, these findings elucidate how cognitive maps are structured into distinct geometric representations to support abstraction and generalization while maintaining memory specificity.

Working memory control dynamics follow principles of spatial computing

Working memory (WM) allows us to remember and selectively control a limited set of items. Neural evidence suggests it is achieved by interactions between bursts of beta and gamma oscillations. However, it is not clear how oscillations, reflecting coherent activity of millions of neurons, can selectively control individual WM items. Here we propose the novel concept of spatial computing where beta and gamma interactions cause item-specific activity to flow spatially across the network during a task. This way, control-related information such as item order is stored in the spatial activity independent of the detailed recurrent connectivity supporting the item-specific activity itself. The spatial flow is in turn reflected in low-dimensional activity shared by many neurons. We verify these predictions by analyzing local field potentials and neuronal spiking. We hypothesize that spatial computing can facilitate generalization and zero-shot learning by utilizing spatial component as an additional information encoding dimension.