Inferotemporal cortex multiplexes behaviorally-relevant target match signals and visual representations in a manner that minimizes their interference

Pinpointing the neural signatures of single-exposure visual recognition memory

Memories of the images that we have seen are thought to be reflected in the reduction of neural responses in high-level visual areas such as inferotemporal (IT) cortex, a phenomenon known as repetition suppression (RS). We challenged this hypothesis with a task that required rhesus monkeys to report whether images were novel or repeated while ignoring variations in contrast, a stimulus attribute that is also known to modulate the overall IT response. The monkeys' behavior was largely contrast invariant, contrary to the predictions of an RS-inspired decoder, which could not distinguish responses to images that are repeated from those that are of lower contrast. However, the monkeys' behavioral patterns were well predicted by a linearly decodable variant in which the total spike count was corrected for contrast modulation. These results suggest that the IT neural activity pattern that best aligns with single-exposure visual recognition memory behavior is not RS but rather sensory referenced suppression: reductions in IT population response magnitude, corrected for sensory modulation.

Task-specific modulation of PFC activity for matching-rule governed decision-making

Storing information from incoming stimuli in working memory (WM) is essential for decision-making. The prefrontal cortex (PFC) plays a key role to support this process. Previous studies have characterized different neuronal populations in the PFC for working memory judgements based on whether an originally presented stimulus matches a subsequently presented one (matching-rule decision-making). However, much remains to be understood about this mechanism at the population level of PFC neurons. Here, we hypothesized differences in processing of feature vs. spatial WM within the PFC during a matching-rule decision-making task. To test this hypothesis, the modulation of neural activity within the PFC during two types of decision-making tasks (spatial WM and feature WM) in comparison to a passive fixation task was determined. We discovered that neural population-level activity within the PFC is different for the match vs. non-match condition exclusively in the case of the feature-specific decision-making task. For this task, the non-match condition exhibited a greater firing rate and lower trial-to-trial variability in spike count compared to the feature-match condition. Furthermore, the feature-match condition exhibited lower variability compared to the spatial-match condition. This was accompanied by a faster behavioral response time for the feature-match compared to the spatial-match WM task. We attribute this lower across-trial spiking variability and behavioral response time to a higher task-relevant attentional level in the feature WM compared to the spatial WM task. The findings support our hypothesis for task-specific differences in the processing of feature vs. spatial WM within the PFC. This also confirms the general conclusion that PFC neurons play an important role during the process of matching-rule governed decision-making.

The integration of visual and target signals in V4 and IT during visual object search

Searching for a specific visual object requires our brain to compare the items in view with a remembered representation of the sought target to determine whether a target match is present. This comparison is thought to be implemented, in part, via the combination of top-down modulations reflecting target identity with feed-forward visual representations. However, it remains unclear whether top-down signals are integrated at a single locus within the ventral visual pathway (e.g., V4) or at multiple stages [e.g., both V4 and inferotemporal cortex (IT)]. To investigate, we recorded neural responses in V4 and IT as rhesus monkeys performed a task that required them to identify when a target object appeared across variation in position, size, and background context. We found nonvisual, task-specific signals in both V4 and IT. To evaluate whether V4 was the only locus for the integration of top-down signals, we evaluated several feed-forward accounts of processing from V4 to IT, including a model in which IT preferentially sampled from the best V4 units and a model that allowed for nonlinear IT computation. IT task-specific modulation was not accounted for by any of these feed-forward descriptions, suggesting that during object search, top-down signals are integrated directly within IT.NEW & NOTEWORTHY To find specific objects, the brain must integrate top-down, target-specific signals with visual information about objects in view. However, the exact route of this integration in the ventral visual pathway is unclear. In the first study to systematically compare V4 and inferotemporal cortex (IT) during an invariant object search task, we demonstrate that top-down signals found in IT cannot be described as being inherited from V4 but rather must be integrated directly within IT itself.

Decoding and encoding (de)mixed population responses

A central tenet of neuroscience is that the brain works through large populations of interacting neurons. With recent advances in recording techniques, the inner working of these populations has come into full view. Analyzing the resulting large-scale data sets is challenging because of the often complex and 'mixed' dependency of neural activities on experimental parameters, such as stimuli, decisions, or motor responses. Here we review recent insights gained from analyzing these data with dimensionality reduction methods that 'demix' these dependencies. We demonstrate that the mappings from (carefully chosen) experimental parameters to population activities appear to be typical and stable across tasks, brain areas, and animals, and are often identifiable by linear methods. By considering when and why dimensionality reduction and demixing work well, we argue for a view of population coding in which populations represent (demixed) latent signals, corresponding to stimuli, decisions, motor responses, and so on. These latent signals are encoded into neural population activity via non-linear mappings and decoded via linear readouts. We explain how such a scheme can facilitate the propagation of information across cortical areas, and we review neural network architectures that can reproduce the encoding and decoding of latent signals in population activities. These architectures promise a link from the biophysics of single neurons to the activities of neural populations.

Rethinking assumptions about how trial and nuisance variability impact neural task performance in a fast-processing regime

Task performance is determined not only by the amount of task-relevant signal present in our brains but also by the presence of noise, which can arise from multiple sources. Internal noise, or "trial variability," manifests as trial-by-trial variations in neural responses under seemingly identical conditions. External factors can also translate into noise, particularly when a task requires extraction of a particular type of information from our environment amid changes in other task-irrelevant "nuisance" parameters. To better understand how signal, trial variability, and nuisance variability combine to determine neural task performance, we explored their interactions, both in simulation and when applied to recorded neural data. This exploration revealed that trial variability is typically larger than a neuron's task-relevant signal for tasks with fast reaction times, where spike count integration windows are short. In this low signal-to-trial variability regime, nuisance variability has the counterintuitive property of having a negligible impact on single-neuron task performance, even when it dominates the task-relevant signal. The inconsequential impact of nuisance variability on individual neurons also extends to descriptions of population performance, under the assumption that both trial and nuisance variability are uncorrelated between neurons. These results demonstrate that some basic intuitions about neural coding are misguided in the context of a fast-processing, low-spike-count regime. NEW & NOTEWORTHY Many everyday tasks require us to extract specific information from our environment while ignoring other things. When the neurons in our brains that carry task-relevant signals are also modulated by task-irrelevant "nuisance" information, nuisance modulation is expected to act as performance-limiting noise. Using both simulated and recorded neural data, we demonstrate that these intuitions are misguided when the brain operates in a fast-processing, low-spike-count regime, where nuisance variability is largely inconsequential for performance.