Sharp emergence of feature-selective sustained activity along the dorsal visual pathway

Active information maintenance in working memory by a sensory cortex

Working memory is a critical brain function for maintaining and manipulating information over delay periods of seconds. It is debated whether delay-period neural activity in sensory regions is important for the active maintenance of information during the delay period. Here, we tackle this question by examining the anterior piriform cortex (APC), an olfactory sensory cortex, in head-fixed mice performing several olfactory working memory tasks. Active information maintenance is necessary in these tasks, especially in a dual-task paradigm in which mice are required to perform another distracting task while actively maintaining information during the delay period. Optogenetic suppression of neuronal activity in APC during the delay period impaired performance in all the tasks. Furthermore, electrophysiological recordings revealed that APC neuronal populations encoded odor information in the delay period even with an intervening distracting task. Thus, delay activity in APC is important for active information maintenance in olfactory working memory.

A multimodal pathway including the basal ganglia in the feline brain

The purpose of this paper is to give an overview of our present knowledge about the feline tecto-thalamo-basal ganglia cortical sensory pathway. We reviewed morphological and electrophysiological studies of the cortical areas, located in ventral bank of the anterior ectosylvian sulcus as well as the region of the insular cortex, the suprageniculate nucleus of the thalamus, caudate nucleus, and the substantia nigra. Microelectrode studies revealed common receptive field properties in all these structures. The receptive fields were extremely large and multisensory, with pronounced sensitivity to motion of visual stimuli. They often demonstrated directional and velocity selectivity. Preference for small visual stimuli was also a frequent finding. However, orientation sensitivity was absent. It became obvious that the structures of the investigated sensory loop exhibit a unique kind of information processing, not found anywhere else in the feline visual system.

A Flexible Model of Working Memory

Working memory is fundamental to cognition, allowing one to hold information "in mind." A defining characteristic of working memory is its flexibility: we can hold anything in mind. However, typical models of working memory rely on finely tuned, content-specific attractors to persistently maintain neural activity and therefore do not allow for the flexibility observed in behavior. Here, we present a flexible model of working memory that maintains representations through random recurrent connections between two layers of neurons: a structured "sensory" layer and a randomly connected, unstructured layer. As the interactions are untuned with respect to the content being stored, the network maintains any arbitrary input. However, in our model, this flexibility comes at a cost: the random connections overlap, leading to interference between representations and limiting the memory capacity of the network. Additionally, our model captures several other key behavioral and neurophysiological characteristics of working memory.

Prefrontal Contributions to Attention and Working Memory

The processes of attention and working memory are conspicuously interlinked, suggesting that they may involve overlapping neural mechanisms. Working memory (WM) is the ability to maintain information in the absence of sensory input. Attention is the process by which a specific target is selected for further processing, and neural resources directed toward that target. The content of WM can be used to direct attention, and attention can in turn determine which information is encoded into WM. Here we discuss the similarities between attention and WM and the role prefrontal cortex (PFC) plays in each. First, at the theoretical level, we describe how attention and WM can both rely on models based on attractor states. Then we review the evidence for an overlap between the areas involved in both functions, especially the frontal eye field (FEF) portion of the prefrontal cortex. We also discuss similarities between the neural changes in visual areas observed during attention and WM. At the cellular level, we review the literature on the role of prefrontal DA in both attention and WM at the behavioral and neural levels. Finally, we summarize the anatomical evidence for an overlap between prefrontal mechanisms involved in attention and WM. Altogether, a summary of pharmacological, electrophysiological, behavioral, and anatomical evidence for a contribution of the FEF part of prefrontal cortex to attention and WM is provided.

Concurrent influence of top-down and bottom-up inputs on correlated activity of Macaque extrastriate neurons

Correlations between neurons can profoundly impact the information encoding capacity of a neural population. We studied how maintenance of visuospatial information affects correlated activity in visual areas by recording the activity of neurons in visual area MT of rhesus macaques during a spatial working memory task. Correlations between MT neurons depended upon the spatial overlap between neurons’ receptive fields. These correlations were influenced by the content of working memory, but the effect of a top-down memory signal differed in the presence or absence of bottom-up visual input. Neurons representing the same area of space showed increased correlations when remembering a location in their receptive fields in the absence of visual input, but decreased correlations in the presence of a visual stimulus. This set of results reveals the correlating nature of top-down signals influencing visual areas and uncovers how such a correlating signal, in interaction with bottom-up information, could enhance sensory representations.

Changes in correlated activity of neurons are believed to influence their information coding capacity. Here, the authors show how top-down and bottom-up inputs and their interaction differentially alter the correlated activity of neurons in extrastriate cortex

Working Memory 2.0

Working memory is the fundamental function by which we break free from reflexive input-output reactions to gain control over our own thoughts. It has two types of mechanisms: online maintenance of information and its volitional or executive control. Classic models proposed persistent spiking for maintenance but have not explicitly addressed executive control. We review recent theoretical and empirical studies that suggest updates and additions to the classic model. Synaptic weight changes between sparse bursts of spiking strengthen working memory maintenance. Executive control acts via interplay between network oscillations in gamma (30-100 Hz) in superficial cortical layers (layers 2 and 3) and alpha and beta (10-30 Hz) in deep cortical layers (layers 5 and 6). Deep-layer alpha and beta are associated with top-down information and inhibition. It regulates the flow of bottom-up sensory information associated with superficial layer gamma. We propose that interactions between different rhythms in distinct cortical layers underlie working memory maintenance and its volitional control.

Introducing a Comprehensive Framework to Measure Spike-LFP Coupling

Measuring the coupling of single neuron's spiking activities to the local field potentials (LFPs) is a method to investigate neuronal synchronization. The most important synchronization measures are phase locking value (PLV), spike field coherence (SFC), pairwise phase consistency (PPC), and spike-triggered correlation matrix synchronization (SCMS). Synchronization is generally quantified using the PLV and SFC. PLV and SFC methods are either biased on the spike rates or the number of trials. To resolve these problems the PPC measure has been introduced. However, there are some shortcomings associated with the PPC measure which is unbiased only for very high spike rates. However evaluating spike-LFP phase coupling (SPC) for short trials or low number of spikes is a challenge in many studies. Lastly, SCMS measures the correlation in terms of phase in regions around the spikes inclusive of the non-spiking events which is the major difference between SCMS and SPC. This study proposes a new framework for predicting a more reliable SPC by modeling and introducing appropriate machine learning algorithms namely least squares, Lasso, and neural networks algorithms where through an initial trend of the spike rates, the ideal SPC is predicted for neurons with low spike rates. Furthermore, comparing the performance of these three algorithms shows that the least squares approach provided the best performance with a correlation of 0.99214 and R² of 0.9563 in the training phase, and correlation of 0.95969 and R² of 0.8842 in the test phase. Hence, the results show that the proposed framework significantly enhances the accuracy and provides a bias-free basis for small number of spikes for SPC as compared to the conventional methods such as PLV method. As such, it has the general ability to correct for the bias on the number of spike rates.

Dynamic population coding and its relationship to working memory

For over 45 years, neuroscientists have conducted experiments aimed at understanding the neural basis of working memory. Early results examining individual neurons highlighted that information is stored in working memory in persistent sustained activity where neurons maintained elevated firing rates over extended periods of time. However, more recent work has emphasized that information is often stored in working memory in dynamic population codes, where different neurons contain information at different periods in time. In this paper, I review findings that show that both sustained activity as well as dynamic codes are present in the prefrontal cortex and other regions during memory delay periods. I also review work showing that dynamic codes are capable of supporting working memory and that such dynamic codes could easily be "readout" by downstream regions. Finally, I discuss why dynamic codes could be useful for enabling animals to solve tasks that involve working memory. Although additional work is still needed to know definitively whether dynamic coding is critical for working memory, the findings reviewed here give insight into how different codes could contribute to working memory, which should be useful for guiding future research.

Auditory modulation of spiking activity and local field potentials in area MT does not appear to underlie an audiovisual temporal illusion

The timing of brief stationary sounds has been shown to alter the perceived speed of visual apparent motion (AM), presumably by altering the perceived timing of the individual frames of the AM stimuli and/or the duration of the interstimulus intervals (ISIs) between those frames. To investigate the neural correlates of this "temporal ventriloquism" illusion, we recorded spiking and local field potential (LFP) activity from the middle temporal area (area MT) in awake, fixating macaques. We found that the spiking activity of most MT neurons (but not the LFP) was tuned for the ISI/speed (these parameters covaried) of our AM stimuli but that auditory timing had no effect on that tuning. We next asked whether the predicted changes in perceived timing were reflected in the timing of neuronal responses to the individual frames of the AM stimuli. Although spiking dynamics were significantly, if weakly, affected by auditory timing in a minority of neurons, the timing of spike responses did not systematically mirror the predicted perception of stimuli. Conversely, the duration of LFP responses in β- and γ-frequency bands was qualitatively consistent with human perceptual reports. We discovered, however, that LFP responses to auditory stimuli presented alone were robust and that responses to audiovisual stimuli were predicted by the linear sum of responses to auditory and visual stimuli presented individually. In conclusion, we find evidence of auditory input into area MT but not of the nonlinear audiovisual interactions we had hypothesized to underlie the illusion. NEW & NOTEWORTHY We utilized a set of audiovisual stimuli that elicit an illusion demonstrating "temporal ventriloquism" in visual motion and that have spatiotemporal intervals for which neurons within the middle temporal area are selective. We found evidence of auditory input into the middle temporal area but not of the nonlinear audiovisual interactions underlying this illusion. Our findings suggest that either the illusion was absent in our nonhuman primate subjects or the neuronal correlates of this illusion lie within other areas.

Dissociation of LFP Power and Tuning in the Frontal Cortex during Memory

Working memory, the ability to maintain and manipulate information in the brain, is critical for cognition. During the memory period of spatial memory tasks, neurons in the prefrontal cortex code for memorized locations via persistent, spatially tuned increases in activity. Local field potentials (LFPs) are understood to reflect summed synaptic activity of local neuron populations and may offer a window into network-level processing. We recorded LFPs from areas 8A and 9/46 while two male cynomolgus macaques (Macaca fascicularis) performed a long duration (5.1-15.6 s) memory-guided saccade task. Greater than ∼16 Hz, LFP power was contralaterally tuned throughout the memory period. Yet power for both contralateral and ipsilateral targets fell gradually after the first second of the memory period, dropping below baseline after a few seconds. Our results dissociate absolute LFP power from mnemonic tuning and are consistent with modeling work that suggests that decreasing synchronization within a network may improve the stability of memory coding.SIGNIFICANCE STATEMENT The frontal cortex is an important site for working memory. There, individual neurons reflect memorized information with selective increases in activity, but how collections of neurons work together to achieve memory is not well understood. In this work, we examined rhythmic electrical activity surrounding these neurons, which may reflect the operation of recurrent circuitry that could underlie memory. This rhythmic activity was spatially tuned with respect to memorized locations as long as memory was tested (∼7.5 s). Surprisingly, however, the overall magnitude of rhythmic activity decreased steadily over this period, dropping below baseline levels after a few seconds. These findings suggest that collections of neurons may actively desynchronize to promote stability in memory circuitry.

Flexible Coding of Visual Working Memory Representations during Distraction

Visual working memory (VWM) recruits a broad network of brain regions, including prefrontal, parietal, and visual cortices. Recent evidence supports a "sensory recruitment" model of VWM, whereby precise visual details are maintained in the same stimulus-selective regions responsible for perception. A key question in evaluating the sensory recruitment model is how VWM representations persist through distracting visual input, given that the early visual areas that putatively represent VWM content are susceptible to interference from visual stimulation.To address this question, we used a functional magnetic resonance imaging inverted encoding model approach to quantitatively assess the effect of distractors on VWM representations in early visual cortex and the intraparietal sulcus (IPS), another region previously implicated in the storage of VWM information. This approach allowed us to reconstruct VWM representations for orientation, both before and after visual interference, and to examine whether oriented distractors systematically biased these representations. In our human participants (both male and female), we found that orientation information was maintained simultaneously in early visual areas and IPS in anticipation of possible distraction, and these representations persisted in the absence of distraction. Importantly, early visual representations were susceptible to interference; VWM orientations reconstructed from visual cortex were significantly biased toward distractors, corresponding to a small attractive bias in behavior. In contrast, IPS representations did not show such a bias. These results provide quantitative insight into the effect of interference on VWM representations, and they suggest a dynamic tradeoff between visual and parietal regions that allows flexible adaptation to task demands in service of VWM.SIGNIFICANCE STATEMENT Despite considerable evidence that stimulus-selective visual regions maintain precise visual information in working memory, it remains unclear how these representations persist through subsequent input. Here, we used quantitative model-based fMRI analyses to reconstruct the contents of working memory and examine the effects of distracting input. Although representations in the early visual areas were systematically biased by distractors, those in the intraparietal sulcus appeared distractor-resistant. In contrast, early visual representations were most reliable in the absence of distraction. These results demonstrate the dynamic, adaptive nature of visual working memory processes, and provide quantitative insight into the ways in which representations can be affected by interference. Further, they suggest that current models of working memory should be revised to incorporate this flexibility.

Customizable cap implants for neurophysiological experimentation

BACKGROUND

Several primate neurophysiology laboratories have adopted acrylic-free, custom-fit cranial implants. These implants are often comprised of titanium or plastic polymers, such as polyether ether ketone (PEEK). Titanium is favored for its mechanical strength and osseointegrative properties whereas PEEK is notable for its lightweight, machinability, and MRI compatibility. Recent titanium/PEEK implants have proven to be effective in minimizing infection and implant failure, thereby prolonging experiments and optimizing the scientific contribution of a single primate.

NEW METHOD

We created novel, customizable PEEK 'cap' implants that contour to the primate's skull. The implants were created using MRI and/or CT data, SolidWorks software and CNC-machining.

RESULTS

Three rhesus macaques were implanted with a PEEK cap implant. Head fixation and chronic recordings were successfully performed. Improvements in design and surgical technique solved issues of granulation tissue formation and headpost screw breakage.

COMPARISON WITH EXISTING METHODS

Primate cranial implants have traditionally been fastened to the skull using acrylic and anchor screws. This technique is prone to skin recession, infection, and implant failure. More recent methods have used imaging data to create custom-fit titanium/PEEK implants with radially extending feet or vertical columns. Compared to our design, these implants are more surgically invasive over time, have less force distribution, and/or do not optimize the utilizable surface area of the skull.

CONCLUSIONS

Our PEEK cap implants served as an effective and affordable means to perform electrophysiological experimentation while reducing surgical invasiveness, providing increased strength, and optimizing useful surface area.

Encoding of Spatial Attention by Primate Prefrontal Cortex Neuronal Ensembles

Single neurons in the primate lateral prefrontal cortex (LPFC) encode information about the allocation of visual attention and the features of visual stimuli. However, how this compares to the performance of neuronal ensembles at encoding the same information is poorly understood. Here, we recorded the responses of neuronal ensembles in the LPFC of two macaque monkeys while they performed a task that required attending to one of two moving random dot patterns positioned in different hemifields and ignoring the other pattern. We found single units selective for the location of the attended stimulus as well as for its motion direction. To determine the coding of both variables in the population of recorded units, we used a linear classifier and progressively built neuronal ensembles by iteratively adding units according to their individual performance (best single units), or by iteratively adding units based on their contribution to the ensemble performance (best ensemble). For both methods, ensembles of relatively small sizes (n < 60) yielded substantially higher decoding performance relative to individual single units. However, the decoder reached similar performance using fewer neurons with the best ensemble building method compared with the best single units method. Our results indicate that neuronal ensembles within the LPFC encode more information about the attended spatial and nonspatial features of visual stimuli than individual neurons. They further suggest that efficient coding of attention can be achieved by relatively small neuronal ensembles characterized by a certain relationship between signal and noise correlation structures.

Working Memory Enhances Cortical Representations via Spatially Specific Coordination of Spike Times

The online maintenance and manipulation of information in working memory (WM) is essential for guiding behavior based on our goals. Understanding how WM alters sensory processing in pursuit of different behavioral objectives is therefore crucial to establish the neural basis of our goal-directed behavior. Here we show that, in the middle temporal (MT) area of rhesus monkeys, the power of the local field potentials in the αβ band (8-25 Hz) increases, reflecting the remembered location and the animal's performance. Moreover, the content of WM determines how coherently MT sites oscillate and how synchronized spikes are relative to these oscillations. These changes in spike timing are not only sufficient to carry sensory and memory information, they can also account for WM-induced sensory enhancement. These results provide a mechanistic-level understanding of how WM alters sensory processing by coordinating the timing of spikes across the neuronal population, enhancing the sensory representation of WM targets.

Processing of Egomotion-Consistent Optic Flow in the Rhesus Macaque Cortex

The cortical network that processes visual cues to self-motion was characterized with functional magnetic resonance imaging in 3 awake behaving macaques. The experimental protocol was similar to previous human studies in which the responses to a single large optic flow patch were contrasted with responses to an array of 9 similar flow patches. This distinguishes cortical regions where neurons respond to flow in their receptive fields regardless of surrounding motion from those that are sensitive to whether the overall image arises from self-motion. In all 3 animals, significant selectivity for egomotion-consistent flow was found in several areas previously associated with optic flow processing, and notably dorsal middle superior temporal area, ventral intra-parietal area, and VPS. It was also seen in areas 7a (Opt), STPm, FEFsem, FEFsac and in a region of the cingulate sulcus that may be homologous with human area CSv. Selectivity for egomotion-compatible flow was never total but was particularly strong in VPS and putative macaque CSv. Direct comparison of results with the equivalent human studies reveals several commonalities but also some differences.

Electrical stimulation of macaque lateral prefrontal cortex modulates oculomotor behavior indicative of a disruption of top-down attention

The lateral prefrontal cortex (lPFC) of primates is hypothesized to be heavily involved in decision-making and selective visual attention. Recent neurophysiological evidence suggests that information necessary for an orchestration of those high-level cognitive factors are indeed represented in the lPFC. However, we know little about the specific contribution of sub-networks within lPFC to the deployment of top-down influences that can be measured in extrastriate visual cortex. Here, we systematically applied electrical stimulations to areas 8Av and 45 of two macaque monkeys performing a concurrent goal-directed saccade task. Despite using currents well above saccadic thresholds of the directly adjacent Frontal Eye Fields (FEF), saccades were only rarely evoked by the stimulation. Instead, two types of behavioral effects were observed: Stimulations of caudal sites in 8Av (close to FEF) shortened or prolonged saccadic reaction times, depending on the task-instructed saccade, while rostral stimulations of 8Av/45 seem to affect the relative attentional weighting of saccade targets as well as saccadic reaction times. These results illuminate important differences in the causal involvement of different sub-networks within the lPFC and are most compatible with a stimulation-induced biasing of stimulus processing that accelerates the detection of saccade targets presented ipsilateral to stimulation through a disruption of contralaterally deployed top-down attention.

Coding of spatial attention priorities and object features in the macaque lateral intraparietal cortex

Primate posterior parietal cortex (PPC) is known to be involved in controlling spatial attention. Neurons in one part of the PPC, the lateral intraparietal area (LIP), show enhanced responses to objects at attended locations. Although many are selective for object features, such as the orientation of a visual stimulus, it is not clear how LIP circuits integrate feature-selective information when providing attentional feedback about behaviorally relevant locations to the visual cortex. We studied the relationship between object feature and spatial attention properties of LIP cells in two macaques by measuring the cells' orientation selectivity and the degree of attentional enhancement while performing a delayed match-to-sample task. Monkeys had to match both the location and orientation of two visual gratings presented separately in time. We found a wide range in orientation selectivity and degree of attentional enhancement among LIP neurons. However, cells with significant attentional enhancement had much less orientation selectivity in their response than cells which showed no significant modulation by attention. Additionally, orientation-selective cells showed working memory activity for their preferred orientation, whereas cells showing attentional enhancement also synchronized with local neuronal activity. These results are consistent with models of selective attention incorporating two stages, where an initial feature-selective process guides a second stage of focal spatial attention. We suggest that LIP contributes to both stages, where the first stage involves orientation-selective LIP cells that support working memory of the relevant feature, and the second stage involves attention-enhanced LIP cells that synchronize to provide feedback on spatial priorities.

Cognitive control during audiovisual working memory engages frontotemporal theta-band interactions

Working memory (WM) maintenance of sensory information has been associated with enhanced cross-frequency coupling between the phase of low frequencies and the amplitude of high frequencies, particularly in medial temporal lobe (MTL) regions. It has been suggested that these WM maintenance processes are controlled by areas of the prefrontal cortex (PFC) via frontotemporal phase synchronisation in low frequency bands. Here, we investigated whether enhanced cognitive control during audiovisual WM as compared to visual WM alone is associated with increased low-frequency phase synchronisation between sensory areas maintaining WM content and areas from PFC. Using magnetoencephalography, we recorded neural oscillatory activity from healthy human participants engaged in an audiovisual delayed-match-to-sample task. We observed that regions from MTL, which showed enhanced theta-beta phase-amplitude coupling (PAC) during the WM delay window, exhibited stronger phase synchronisation within the theta-band (4-7 Hz) to areas from lateral PFC during audiovisual WM as compared to visual WM alone. Moreover, MTL areas also showed enhanced phase synchronisation to temporooccipital areas in the beta-band (20-32 Hz). Our results provide further evidence that a combination of long-range phase synchronisation and local PAC might constitute a mechanism for neuronal communication between distant brain regions and across frequencies during WM maintenance.

Beyond the Status Quo: A Role for Beta Oscillations in Endogenous Content (Re)Activation

Among the rhythms of the brain, oscillations in the beta frequency range (∼13-30 Hz) have been considered the most enigmatic. Traditionally associated with sensorimotor functions, beta oscillations have recently become more broadly implicated in top-down processing, long-range communication, and preservation of the current brain state. Here, we extend and refine these views based on accumulating new findings of content-specific beta-synchronization during endogenous information processing in working memory (WM) and decision making. We characterize such content-specific beta activity as short-lived, flexible network dynamics supporting the endogenous (re)activation of cortical representations. Specifically, we suggest that beta-mediated ensemble formation within and between cortical areas may awake, rather than merely preserve, an endogenous cognitive set in the service of current task demands. This proposal accommodates key aspects of content-specific beta modulations in monkeys and humans, integrates with timely computational models, and outlines a functional role for beta that fits its transient temporal characteristics.

Reevaluating the Sensory Account of Visual Working Memory Storage

Recent human fMRI pattern-decoding studies have highlighted the involvement of sensory areas in visual working memory (VWM) tasks and argue for a sensory account of VWM storage. In this review, evidence is examined from human behavior, fMRI decoding, and transcranial magnetic stimulation (TMS) studies, as well as from monkey neurophysiology studies. Contrary to the prevalent view, the available evidence provides little support for the sensory account of VWM storage. Instead, when the ability to resist distraction and the existence of top-down feedback are taken into account, VWM-related activities in sensory areas seem to reflect feedback signals indicative of VWM storage elsewhere in the brain. Collectively, the evidence shows that prefrontal and parietal regions, rather than sensory areas, play more significant roles in VWM storage.

Feature-Selective Attentional Modulations in Human Frontoparietal Cortex

Control over visual selection has long been framed in terms of a dichotomy between "source" and "site," where top-down feedback signals originating in frontoparietal cortical areas modulate or bias sensory processing in posterior visual areas. This distinction is motivated in part by observations that frontoparietal cortical areas encode task-level variables (e.g., what stimulus is currently relevant or what motor outputs are appropriate), while posterior sensory areas encode continuous or analog feature representations. Here, we present evidence that challenges this distinction. We used fMRI, a roving searchlight analysis, and an inverted encoding model to examine representations of an elementary feature property (orientation) across the entire human cortical sheet while participants attended either the orientation or luminance of a peripheral grating. Orientation-selective representations were present in a multitude of visual, parietal, and prefrontal cortical areas, including portions of the medial occipital cortex, the lateral parietal cortex, and the superior precentral sulcus (thought to contain the human homolog of the macaque frontal eye fields). Additionally, representations in many-but not all-of these regions were stronger when participants were instructed to attend orientation relative to luminance. Collectively, these findings challenge models that posit a strict segregation between sources and sites of attentional control on the basis of representational properties by demonstrating that simple feature values are encoded by cortical regions throughout the visual processing hierarchy, and that representations in many of these areas are modulated by attention.

SIGNIFICANCE STATEMENT

Influential models of visual attention posit a distinction between top-down control and bottom-up sensory processing networks. These models are motivated in part by demonstrations showing that frontoparietal cortical areas associated with top-down control represent abstract or categorical stimulus information, while visual areas encode parametric feature information. Here, we show that multivariate activity in human visual, parietal, and frontal cortical areas encode representations of a simple feature property (orientation). Moreover, representations in several (though not all) of these areas were modulated by feature-based attention in a similar fashion. These results provide an important challenge to models that posit dissociable top-down control and sensory processing networks on the basis of representational properties.

Restoring Latent Visual Working Memory Representations in Human Cortex

Working memory (WM) enables the storage and manipulation of limited amounts of information over short periods. Prominent models posit that increasing the number of remembered items decreases the spiking activity dedicated to each item via mutual inhibition, which irreparably degrades the fidelity of each item's representation. We tested these models by determining if degraded memory representations could be recovered following a post-cue indicating which of several items in spatial WM would be recalled. Using an fMRI-based image reconstruction technique, we identified impaired behavioral performance and degraded mnemonic representations with elevated memory load. However, in several cortical regions, degraded mnemonic representations recovered substantially following a post-cue, and this recovery tracked behavioral performance. These results challenge pure spike-based models of WM and suggest that remembered items are additionally encoded within latent or hidden neural codes that can help reinvigorate active WM representations.

Neuronal population coding of perceived and memorized visual features in the lateral prefrontal cortex

The primate lateral prefrontal cortex (LPFC) encodes visual stimulus features while they are perceived and while they are maintained in working memory. However, it remains unclear whether perceived and memorized features are encoded by the same or different neurons and population activity patterns. Here we record LPFC neuronal activity while monkeys perceive the motion direction of a stimulus that remains visually available, or memorize the direction if the stimulus disappears. We find neurons with a wide variety of combinations of coding strength for perceived and memorized directions: some neurons encode both to similar degrees while others preferentially or exclusively encode either one. Reading out the combined activity of all neurons, a machine-learning algorithm reliably decode the motion direction and determine whether it is perceived or memorized. Our results indicate that a functionally diverse population of LPFC neurons provides a substrate for discriminating between perceptual and mnemonic representations of visual features.

Neurons in the lateral prefrontal cortex are known to encode visual features as well as maintain them in working memory. Here the authors report that LPFC neurons encode both perceived and memorized visual features in diverse combinations and the population activity reliably decodes as well as differentiates between these two representations.

Mnemonic Encoding and Cortical Organization in Parietal and Prefrontal Cortices

Persistent activity within the frontoparietal network is consistently observed during tasks that require working memory. However, the neural circuit mechanisms underlying persistent neuronal encoding within this network remain unresolved. Here, we ask how neural circuits support persistent activity by examining population recordings from posterior parietal (PPC) and prefrontal (PFC) cortices in two male monkeys that performed spatial and motion direction-based tasks that required working memory. While spatially selective persistent activity was observed in both areas, robust selective persistent activity for motion direction was only observed in PFC. Crucially, we find that this difference between mnemonic encoding in PPC and PFC is associated with the presence of functional clustering: PPC and PFC neurons up to ∼700 μm apart preferred similar spatial locations, and PFC neurons up to ∼700 μm apart preferred similar motion directions. In contrast, motion-direction tuning similarity between nearby PPC neurons was much weaker and decayed rapidly beyond ∼200 μm. We also observed a similar association between persistent activity and functional clustering in trained recurrent neural network models embedded with a columnar topology. These results suggest that functional clustering facilitates mnemonic encoding of sensory information.SIGNIFICANCE STATEMENT Working memory refers to our ability to temporarily store and manipulate information. Numerous studies have observed that, during working memory, neurons in higher cortical areas, such as the parietal and prefrontal cortices, mnemonically encode the remembered stimulus. However, several recent studies have failed to observe mnemonic encoding during working memory, raising the question as to why mnemonic encoding is observed during some, but not all, conditions. In this study, we show that mnemonic encoding occurs when a cortical area is organized such that nearby neurons preferentially respond to the same stimulus. This result provides plausible neuronal conditions that allow for mnemonic encoding, and gives us further understanding of the brain's mechanisms that support working memory.

Sustained Activity Encoding Working Memories: Not Fully Distributed

Working memory (WM) is the ability to remember and manipulate information for short time intervals. Recent studies have proposed that sustained firing encoding the contents of WM is ubiquitous across cortical neurons. We review here the collective evidence supporting this claim. A variety of studies report that neurons in prefrontal, parietal, and inferotemporal association cortices show robust sustained activity encoding the location and features of memoranda during WM tasks. However, reports of WM-related sustained activity in early sensory areas are rare, and typically lack stimulus specificity. We propose that robust sustained activity that can support WM coding arises as a property of association cortices downstream from the early stages of sensory processing.

Spatial working memory alters the efficacy of input to visual cortex

Prefrontal cortex modulates sensory signals in extrastriate visual cortex, in part via its direct projections from the frontal eye field (FEF), an area involved in selective attention. We find that working memory-related activity is a dominant signal within FEF input to visual cortex. Although this signal alone does not evoke spiking responses in areas V4 and MT during memory, the gain of visual responses in these areas increases, and neuronal receptive fields expand and shift towards the remembered location, improving the stimulus representation by neuronal populations. These results provide a basis for enhancing the representation of working memory targets and implicate persistent FEF activity as a basis for the interdependence of working memory and selective attention.

Frontal eye field (FEF) is a visual prefrontal area involved in top-down attention. Here the authors report that FEF neurons projecting to V4/MT are persistently active during spatial working memory, and V4/MT neurons show changes in receptive field and gain at the location held in working memory.

Attention to Multiple Objects Facilitates Their Integration in Prefrontal and Parietal Cortex

Selective attention is known to interact with perceptual organization. In visual scenes, individual objects that are distinct and discriminable may occur on their own, or in groups such as a stack of books. The main objective of this study is to probe the neural interaction that occurs between individual objects when attention is directed toward one or more objects. Here we record steady-state visual evoked potentials via electrocorticography to directly assess the responses to individual stimuli and to their interaction. When human participants attend to two adjacent stimuli, prefrontal and parietal cortex shows a selective enhancement of only the neural interaction between stimuli, but not the responses to individual stimuli. When only one stimulus is attended, the neural response to that stimulus is selectively enhanced in prefrontal and parietal cortex. In contrast, early visual areas generally manifest responses to individual stimuli and to their interaction regardless of attentional task, although a subset of the responses is modulated similarly to prefrontal and parietal cortex. Thus, the neural representation of the visual scene as one progresses up the cortical hierarchy becomes more highly task-specific and represents either individual stimuli or their interaction, depending on the behavioral goal. Attention to multiple objects facilitates an integration of objects akin to perceptual grouping.SIGNIFICANCE STATEMENT Individual objects in a visual scene are seen as distinct entities or as parts of a whole. Here we examine how attention to multiple objects affects their neural representation. Previous studies measured single-cell or fMRI responses and obtained only aggregate measures that combined the activity to individual stimuli as well as their potential interaction. Here, we directly measure electrocorticographic steady-state responses corresponding to individual objects and to their interaction using a frequency-tagging technique. Attention to two stimuli increases the interaction component that is a hallmark for perceptual integration of stimuli. Furthermore, this stimulus-specific interaction is represented in prefrontal and parietal cortex in a task-dependent manner.

Visual and presaccadic activity in area 8Ar of the macaque monkey lateral prefrontal cortex

Common trends observed in many visual and oculomotor-related cortical areas include retinotopically organized receptive and movement fields exhibiting a Gaussian shape and increasing size with eccentricity. These trends are demonstrated in the frontal eye fields, many visual areas, and the superior colliculus but have not been thoroughly characterized in prearcuate area 8Ar of the prefrontal cortex. This is important since area 8Ar, located anterior to the frontal eye fields, is more cytoarchitectonically similar to prefrontal areas than premotor areas. Here we recorded the responses of 166 neurons in area 8Ar of two male macaques while the animals made visually guided saccades to a peripheral sine-wave grating stimulus positioned at 1 of 40 possible locations (8 angles along 5 eccentricities). To characterize the neurons' receptive and movement fields, we fit a bivariate Gaussian model to the baseline-subtracted average firing rate during stimulus presentation (early and late visual epochs) and before saccade onset (presaccadic epoch). One hundred twenty-one of one hundred sixty-six neurons showed spatially selective visual and presaccadic responses. Of the visually selective neurons, 76% preferred the contralateral visual hemifield, whereas 24% preferred the ipsilateral hemifield. The angular width of visual and movement-related fields scaled positively with increasing eccentricity. Moreover, responses of neurons with visual receptive fields were modulated by target contrast, exhibiting sigmoid tuning curves that resemble those of visual neurons in upstream areas such as MT and V4. Finally, we found that neurons with receptive fields at similar spatial locations were clustered within the area; however, this organization did not appear retinotopic.NEW & NOTEWORTHY We recorded the responses of neurons in lateral prefrontal area 8Ar of macaques during a visually guided saccade task using multielectrode arrays. Neurons have Gaussian-shaped visual and movement fields in both visual hemifields, with a bias toward the contralateral hemifield. Visual neurons show contrast response functions with sigmoid shapes. Visual neurons tend to cluster at similar locations within the cortical surface; however, this organization does not appear retinotopic.

Correlated variability modifies working memory fidelity in primate prefrontal neuronal ensembles

Neurons in the primate lateral prefrontal cortex (LPFC) encode working memory (WM) representations via sustained firing, a phenomenon hypothesized to arise from recurrent dynamics within ensembles of interconnected neurons. Here, we tested this hypothesis by using microelectrode arrays to examine spike count correlations (r_sc ) in LPFC neuronal ensembles during a spatial WM task. We found a pattern of pairwise r_sc during WM maintenance indicative of stronger coupling between similarly tuned neurons and increased inhibition between dissimilarly tuned neurons. We then used a linear decoder to quantify the effects of the high-dimensional r_sc structure on information coding in the neuronal ensembles. We found that the r_sc structure could facilitate or impair coding, depending on the size of the ensemble and tuning properties of its constituent neurons. A simple optimization procedure demonstrated that near-maximum decoding performance could be achieved using a relatively small number of neurons. These WM-optimized subensembles were more signal correlation (r_signal )-diverse and anatomically dispersed than predicted by the statistics of the full recorded population of neurons, and they often contained neurons that were poorly WM-selective, yet enhanced coding fidelity by shaping the ensemble's r_sc structure. We observed a pattern of r_sc between LPFC neurons indicative of recurrent dynamics as a mechanism for WM-related activity and that the r_sc structure can increase the fidelity of WM representations. Thus, WM coding in LPFC neuronal ensembles arises from a complex synergy between single neuron coding properties and multidimensional, ensemble-level phenomena.

Assessing the Effect of Early Visual Cortex Transcranial Magnetic Stimulation on Working Memory Consolidation

Maintaining visual working memory (VWM) representations recruits a network of brain regions, including the frontal, posterior parietal, and occipital cortices; however, it is unclear to what extent the occipital cortex is engaged in VWM after sensory encoding is completed. Noninvasive brain stimulation data show that stimulation of this region can affect working memory (WM) during the early consolidation time period, but it remains unclear whether it does so by influencing the number of items that are stored or their precision. In this study, we investigated whether single-pulse transcranial magnetic stimulation (spTMS) to the occipital cortex during VWM consolidation affects the quantity or quality of VWM representations. In three experiments, we disrupted VWM consolidation with either a visual mask or spTMS to retinotopic early visual cortex. We found robust masking effects on the quantity of VWM representations up to 200 msec poststimulus offset and smaller, more variable effects on WM quality. Similarly, spTMS decreased the quantity of VWM representations, but only when it was applied immediately following stimulus offset. Like visual masks, spTMS also produced small and variable effects on WM precision. The disruptive effects of both masks and TMS were greatly reduced or entirely absent within 200 msec of stimulus offset. However, there was a reduction in swap rate across all time intervals, which may indicate a sustained role of the early visual cortex in maintaining spatial information.

Posterior α EEG Dynamics Dissociate Current from Future Goals in Working Memory-Guided Visual Search

Current models of visual search assume that search is guided by an active visual working memory representation of what we are currently looking for. This attentional template for currently relevant stimuli can be dissociated from accessory memory representations that are only needed prospectively, for a future task, and that should be prevented from guiding current attention. However, it remains unclear what electrophysiological mechanisms dissociate currently relevant (serving upcoming selection) from prospectively relevant memories (serving future selection). We measured EEG of 20 human subjects while they performed two consecutive visual search tasks. Before the search tasks, a cue instructed observers which item to look for first (current template) and which second (prospective template). During the delay leading up to the first search display, we found clear suppression of α band (8-14 Hz) activity in regions contralateral to remembered items, comprising both local power and interregional phase synchronization within a posterior parietal network. Importantly, these lateralization effects were stronger when the memory item was currently relevant (i.e., for the first search) compared with when it was prospectively relevant (i.e., for the second search), consistent with current templates being prioritized over future templates. In contrast, event-related potential analysis revealed that the contralateral delay activity was similar for all conditions, suggesting no difference in storage. Together, these findings support the idea that posterior α oscillations represent a state of increased processing or excitability in task-relevant cortical regions, and reflect enhanced cortical prioritization of memory representations that serve as a current selection filter.SIGNIFICANCE STATEMENT Our days are filled with looking for relevant objects while ignoring irrelevant visual information. Such visual search activity is thought to be driven by current goals activated in working memory. However, working memory not only serves current goals, but also future goals, with differential impact upon visual selection. Little is known about how the brain differentiates between current and future goals. Here we show, for the first time, that modulations of brain oscillations in the EEG α frequency band in posterior cortex can dissociate current from future search goals in working memory. Moreover, the dynamics of these oscillations uncover how we flexibly switch focus between memory representations. Together, we reveal how the brain assigns priority for selection.

Layer-specificity in the effects of attention and working memory on activity in primary visual cortex

Neuronal activity in early visual cortex depends on attention shifts but the contribution to working memory has remained unclear. Here, we examine neuronal activity in the different layers of the primary visual cortex (V1) in an attention-demanding and a working memory task. A current-source density analysis reveales top-down inputs in the superficial layers and layer 5, and an increase in neuronal firing rates most pronounced in the superficial and deep layers and weaker in input layer 4. This increased activity is strongest in the attention task but it is also highly reliable during working memory delays. A visual mask erases the V1 memory activity, but it reappeares at a later point in time. These results provide new insights in the laminar circuits involved in the top-down modulation of activity in early visual cortex in the presence and absence of visual stimuli.

The Distributed Nature of Working Memory

Studies in humans and non-human primates have provided evidence for storage of working memory contents in multiple regions ranging from sensory to parietal and prefrontal cortex. We discuss potential explanations for these distributed representations: (i) features in sensory regions versus prefrontal cortex differ in the level of abstractness and generalizability; and (ii) features in prefrontal cortex reflect representations that are transformed for guidance of upcoming behavioral actions. We propose that the propensity to produce persistent activity is a general feature of cortical networks. Future studies may have to shift focus from asking where working memory can be observed in the brain to how a range of specialized brain areas together transform sensory information into a delayed behavioral response.

Neural mechanisms of information storage in visual short-term memory

The capacity to briefly memorize fleeting sensory information supports visual search and behavioral interactions with relevant stimuli in the environment. Traditionally, studies investigating the neural basis of visual short term memory (STM) have focused on the role of prefrontal cortex (PFC) in exerting executive control over what information is stored and how it is adaptively used to guide behavior. However, the neural substrates that support the actual storage of content-specific information in STM are more controversial, with some attributing this function to PFC and others to the specialized areas of early visual cortex that initially encode incoming sensory stimuli. In contrast to these traditional views, I will review evidence suggesting that content-specific information can be flexibly maintained in areas across the cortical hierarchy ranging from early visual cortex to PFC. While the factors that determine exactly where content-specific information is represented are not yet entirely clear, recognizing the importance of task-demands and better understanding the operation of non-spiking neural codes may help to constrain new theories about how memories are maintained at different resolutions, across different timescales, and in the presence of distracting information.

Inferring Cortical Variability from Local Field Potentials

The responses of sensory neurons can be quite different to repeated presentations of the same stimulus. Here, we demonstrate a direct link between the trial-to-trial variability of cortical neuron responses and network activity that is reflected in local field potentials (LFPs). Spikes and LFPs were recorded with a multielectrode array from the middle temporal (MT) area of the visual cortex of macaques during the presentation of continuous optic flow stimuli. A maximum likelihood-based modeling framework was used to predict single-neuron spiking responses using the stimulus, the LFPs, and the activity of other recorded neurons. MT neuron responses were strongly linked to gamma oscillations (maximum at 40 Hz) as well as to lower-frequency delta oscillations (1-4 Hz), with consistent phase preferences across neurons. The predicted modulation associated with the LFP was largely complementary to that driven by visual stimulation, as well as the activity of other neurons, and accounted for nearly half of the trial-to-trial variability in the spiking responses. Moreover, the LFP model predictions accurately captured the temporal structure of noise correlations between pairs of simultaneously recorded neurons, and explained the variation in correlation magnitudes observed across the population. These results therefore identify signatures of network activity related to the variability of cortical neuron responses, and suggest their central role in sensory cortical function.

SIGNIFICANCE STATEMENT

The function of sensory neurons is nearly always cast in terms of representing sensory stimuli. However, recordings from visual cortex in awake animals show that a large fraction of neural activity is not predictable from the stimulus. We show that this variability is predictable given the simultaneously recorded measures of network activity, local field potentials. A model that combines elements of these signals with the stimulus processing of the neuron can predict neural responses dramatically better than current models, and can predict the structure of correlations across the cortical population. In identifying ways to understand stimulus processing in the context of ongoing network activity, this work thus provides a foundation to understand the role of sensory cortex in combining sensory and cognitive variables.

Early Visual Cortex as a Multiscale Cognitive Blackboard

Neurons in early visual cortical areas not only represent incoming visual information but are also engaged by higher level cognitive processes, including attention, working memory, imagery, and decision-making. Are these cognitive effects an epiphenomenon or are they functionally relevant for these mental operations? We review evidence supporting the hypothesis that the modulation of activity in early visual areas has a causal role in cognition. The modulatory influences allow the early visual cortex to act as a multiscale cognitive blackboard for read and write operations by higher visual areas, which can thereby efficiently exchange information. This blackboard architecture explains how the activity of neurons in the early visual cortex contributes to scene segmentation and working memory, and relates to the subject's inferences about the visual world. The architecture also has distinct advantages for the processing of visual routines that rely on a number of sequentially executed processing steps.

Visual motion serves but is not under the purview of the dorsal pathway

Visual motion processing is often attributed to the dorsal visual pathway despite visual motion's involvement in almost all visual functions. Furthermore, some visual motion tasks critically depend on the structural integrity of regions outside the dorsal pathway. Here, based on numerous studies, I propose that visual motion signals are swiftly transmitted via multiple non-hierarchical routes to primary motion-dedicated processing regions (MT/V5 and MST) that are not part of the dorsal pathway, and then propagated to a multiplicity of brain areas according to task demands, reaching these regions earlier than the dorsal/ventral hierarchical flow. This not only places MT/V5 at the same or even earlier visual processing stage as that of V1, but can also elucidate many findings with implications to visual awareness. While the integrity of the non-hierarchical motion pathway is necessary for all visual motion perception, it is insufficient on its own, and the transfer of visual motion signals to additional brain areas is crucial to allow the different motion perception tasks (e.g. optic flow, visuo-vestibular balance, movement observation, dynamic form detection and perception, and even reading). I argue that this lateral visual motion pathway can be distinguished from the dorsal pathway not only based on faster response latencies and distinct anatomical connections, but also based on its full field representation. I also distinguish between this primary lateral visual motion pathway sensitive to all motion in the visual field, and a much less investigated optic flow sensitive medial processing pathway (from V1 to V6 and V6A) that appears to be part of the dorsal pathway. Multiple additional predictions are provided that allow testing this proposal and distinguishing between the visual pathways.

The neuroscience of working memory capacity and training

Working memory - the ability to maintain and manipulate information over a period of seconds - is a core component of higher cognitive functions. The storage capacity of working memory is limited but can be expanded by training, and evidence of the neural mechanisms underlying this effect is accumulating. Human imaging studies and neurophysiological recordings in non-human primates, together with computational modelling studies, reveal that training increases the activity of prefrontal neurons and the strength of connectivity in the prefrontal cortex and between the prefrontal and parietal cortex. Dopaminergic transmission could have a facilitatory role. These changes more generally inform us of the plasticity of higher cognitive functions.

A Code for Cross-Modal Working Memory

In this issue of Neuron, Vergara et al. (2016) report that neurons in the pre-supplementary motor area represent the frequency of tactile and auditory stimuli held in working memory. Single neurons encode both types of information by using the same representation for both modalities.

Transitions between Multiband Oscillatory Patterns Characterize Memory-Guided Perceptual Decisions in Prefrontal Circuits

Neuronal activity in the lateral prefrontal cortex (LPFC) reflects the structure and cognitive demands of memory-guided sensory discrimination tasks. However, we still do not know how neuronal activity articulates in network states involved in perceiving, remembering, and comparing sensory information during such tasks. Oscillations in local field potentials (LFPs) provide fingerprints of such network dynamics. Here, we examined LFPs recorded from LPFC of macaques while they compared the directions or the speeds of two moving random-dot patterns, S1 and S2, separated by a delay. LFP activity in the theta, beta, and gamma bands tracked consecutive components of the task. In response to motion stimuli, LFP theta and gamma power increased, and beta power decreased, but showed only weak motion selectivity. In the delay, LFP beta power modulation anticipated the onset of S2 and encoded the task-relevant S1 feature, suggesting network dynamics associated with memory maintenance. After S2 onset the difference between the current stimulus S2 and the remembered S1 was strongly reflected in broadband LFP activity, with an early sensory-related component proportional to stimulus difference and a later choice-related component reflecting the behavioral decision buildup. Our results demonstrate that individual LFP bands reflect both sensory and cognitive processes engaged independently during different stages of the task. This activation pattern suggests that during elementary cognitive tasks, the prefrontal network transitions dynamically between states and that these transitions are characterized by the conjunction of LFP rhythms rather than by single LFP bands.

SIGNIFICANCE STATEMENT

Neurons in the brain communicate through electrical impulses and coordinate this activity in ensembles that pulsate rhythmically, very much like musical instruments in an orchestra. These rhythms change with "brain state," from sleep to waking, but also signal with different oscillation frequencies rapid changes between sensory and cognitive processing. Here, we studied rhythmic electrical activity in the monkey prefrontal cortex, an area implicated in working memory, decision making, and executive control. Monkeys had to identify and remember a visual motion pattern and compare it to a second pattern. We found orderly transitions between rhythmic activity where the same frequency channels were active in all ongoing prefrontal computations. This supports prefrontal circuit dynamics that transitions rapidly between complex rhythmic patterns during structured cognitive tasks.

Turning Symbolic: The Representation of Motion Direction in Working Memory

What happens to the representation of a moving stimulus when it is no longer present and its motion direction has to be maintained in working memory (WM)? Is the initial, sensorial representation maintained during the delay period or is there another representation, at a higher level of abstraction? It is also feasible that multiple representations may co-exist in WM, manifesting different facets of sensory and more abstract features. To that end, we investigated the mnemonic representation of motion direction in a series of three psychophysical experiments, using a delayed motion-discrimination task (relative clockwise∖counter-clockwise judgment). First, we show that a change in the dots’ contrast polarity does not hamper performance. Next, we demonstrate that performance is unaffected by relocation of the Test stimulus in either retinotopic or spatiotopic coordinate frames. Finally, we show that an arrow-shaped cue presented during the delay interval between the Sample and Test stimulus, strongly biases performance toward the direction of the arrow, although the cue itself is non-informative (it has no predictive value of the correct answer). These results indicate that the representation of motion direction in WM could be independent of the physical features of the stimulus (polarity or position) and has non-sensorial abstract qualities. It is plausible that an abstract mnemonic trace might be activated alongside a more basic, analog representation of the stimulus. We speculate that the specific sensitivity of the mnemonic representation to the arrow-shaped symbol may stem from the long term learned association between direction and the hour in the clock.