1
|
Johnston WJ, Freedman DJ. Redundant representations are required to disambiguate simultaneously presented complex stimuli. PLoS Comput Biol 2023; 19:e1011327. [PMID: 37556470 PMCID: PMC10442167 DOI: 10.1371/journal.pcbi.1011327] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2023] [Revised: 08/21/2023] [Accepted: 07/04/2023] [Indexed: 08/11/2023] Open
Abstract
A pedestrian crossing a street during rush hour often looks and listens for potential danger. When they hear several different horns, they localize the cars that are honking and decide whether or not they need to modify their motor plan. How does the pedestrian use this auditory information to pick out the corresponding cars in visual space? The integration of distributed representations like these is called the assignment problem, and it must be solved to integrate distinct representations across but also within sensory modalities. Here, we identify and analyze a solution to the assignment problem: the representation of one or more common stimulus features in pairs of relevant brain regions-for example, estimates of the spatial position of cars are represented in both the visual and auditory systems. We characterize how the reliability of this solution depends on different features of the stimulus set (e.g., the size of the set and the complexity of the stimuli) and the details of the split representations (e.g., the precision of each stimulus representation and the amount of overlapping information). Next, we implement this solution in a biologically plausible receptive field code and show how constraints on the number of neurons and spikes used by the code force the brain to navigate a tradeoff between local and catastrophic errors. We show that, when many spikes and neurons are available, representing stimuli from a single sensory modality can be done more reliably across multiple brain regions, despite the risk of assignment errors. Finally, we show that a feedforward neural network can learn the optimal solution to the assignment problem, even when it receives inputs in two distinct representational formats. We also discuss relevant results on assignment errors from the human working memory literature and show that several key predictions of our theory already have support.
Collapse
Affiliation(s)
- W. Jeffrey Johnston
- Graduate Program in Computational Neuroscience and the Department of Neurobiology, The University of Chicago, Chicago, Illinois, United States of America
- Center for Theoretical Neuroscience and Mortimer B. Zuckerman Mind, Brain and Behavior Institute, Columbia University, New York, New York, United States of America
| | - David J. Freedman
- Graduate Program in Computational Neuroscience and the Department of Neurobiology, The University of Chicago, Chicago, Illinois, United States of America
- Neuroscience Institute, The University of Chicago, Chicago, Illinois, United States of America
| |
Collapse
|
2
|
Black J, Nozari N. Precision of phonological errors in aphasia supports resource models of phonological working memory in language production. Cogn Neuropsychol 2023; 40:1-24. [PMID: 37127940 PMCID: PMC10336978 DOI: 10.1080/02643294.2023.2206012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2022] [Revised: 04/12/2023] [Accepted: 04/18/2023] [Indexed: 05/03/2023]
Abstract
Working memory (WM) is critical for many cognitive functions including language production. A key feature of WM is its capacity limitation. Two models have been proposed to account for such capacity limitation: slot models and resource models. In recent years, resource models have found support in both visual and auditory perception, but do they also extend to production? We investigate this by analyzing sublexical errors from four individuals with aphasia. Using tools from computational linguistics, we first define the concept of "precision" of sublexical errors. We then demonstrate that such precision decreases with increased working memory load, i.e., word length, as predicted by resource models. Finally, we rule out alternative accounts of this effect, such as articulatory simplification. These data provide the first evidence for the applicability of the resource model to production and further point to the generalizability of this account as a model of resource division in WM.
Collapse
Affiliation(s)
- Jenah Black
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA, USA
- Center for the Neural Basis of Cognition (CNBC), Pittsburgh, PA, USA
| | - Nazbanou Nozari
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA, USA
- Center for the Neural Basis of Cognition (CNBC), Pittsburgh, PA, USA
| |
Collapse
|
3
|
Taubert J, Wardle SG, Tardiff CT, Patterson A, Yu D, Baker CI. Clutter Substantially Reduces Selectivity for Peripheral Faces in the Macaque Brain. J Neurosci 2022; 42:6739-6750. [PMID: 35868861 PMCID: PMC9436017 DOI: 10.1523/jneurosci.0232-22.2022] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2022] [Revised: 04/29/2022] [Accepted: 06/06/2022] [Indexed: 11/21/2022] Open
Abstract
According to a prominent view in neuroscience, visual stimuli are coded by discrete cortical networks that respond preferentially to specific categories, such as faces or objects. However, it remains unclear how these category-selective networks respond when viewing conditions are cluttered, i.e., when there is more than one stimulus in the visual field. Here, we asked three questions: (1) Does clutter reduce the response and selectivity for faces as a function of retinal location? (2) Is the preferential response to faces uniform across the visual field? And (3) Does the ventral visual pathway encode information about the location of cluttered faces? We used fMRI to measure the response of the face-selective network in awake, fixating macaques (two female, five male). Across a series of four experiments, we manipulated the presence and absence of clutter, as well as the location of the faces relative to the fovea. We found that clutter reduces the response to peripheral faces. When presented in isolation, without clutter, the selectivity for faces is fairly uniform across the visual field, but, when clutter is present, there is a marked decrease in the selectivity for peripheral faces. We also found no evidence of a contralateral visual field bias when faces were presented in clutter. Nonetheless, multivariate analyses revealed that the location of cluttered faces could be decoded from the multivoxel response of the face-selective network. Collectively, these findings demonstrate that clutter blunts the selectivity of the face-selective network to peripheral faces, although information about their retinal location is retained.SIGNIFICANCE STATEMENT Numerous studies that have measured brain activity in macaques have found visual regions that respond preferentially to faces. Although these regions are thought to be essential for social behavior, their responses have typically been measured while faces were presented in isolation, a situation atypical of the real world. How do these regions respond when faces are presented with other stimuli? We report that, when clutter is present, the preferential response to foveated faces is spared but preferential response to peripheral faces is reduced. Our results indicate that the presence of clutter changes the response of the face-selective network.
Collapse
Affiliation(s)
- Jessica Taubert
- Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, Maryland 20814
- School of Psychology, The University of Queensland, Brisbane, Queensland 4072, Australia
| | - Susan G Wardle
- Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, Maryland 20814
| | - Clarissa T Tardiff
- Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, Maryland 20814
| | - Amanda Patterson
- Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, Maryland 20814
| | - David Yu
- Neurophysiology Imaging Facility, National Institutes of Health, Bethesda, Maryland 20814
| | - Chris I Baker
- Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, Maryland 20814
| |
Collapse
|
4
|
Gross S. Perceptual consciousness and cognitive access from the perspective of capacity-unlimited working memory. Philos Trans R Soc Lond B Biol Sci 2019; 373:rstb.2017.0343. [PMID: 30061457 DOI: 10.1098/rstb.2017.0343] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/21/2018] [Indexed: 01/23/2023] Open
Abstract
Theories of consciousness divide over whether perceptual consciousness is rich or sparse in specific representational content and whether it requires cognitive access. These two issues are often treated in tandem because of a shared assumption that the representational capacity of cognitive access is fairly limited. Recent research on working memory challenges this shared assumption. This paper argues that abandoning the assumption undermines post-cue-based 'overflow' arguments, according to which perceptual consciousness is rich and does not require cognitive access. Abandoning it also dissociates the rich/sparse debate from the access question. The paper then explores attempts to reformulate overflow theses in ways that do not require the assumption of limited capacity. Finally, it discusses the problem of relating seemingly non-probabilistic perceptual consciousness to the probabilistic representations posited by the models that challenge conceptions of cognitive access as capacity-limited.This article is part of the theme issue 'Perceptual consciousness and cognitive access'.
Collapse
Affiliation(s)
- Steven Gross
- Department of Philosophy, Johns Hopkins University, 3400 N. Charles St., Baltimore, MD 21218, USA
| |
Collapse
|
5
|
Keemink SW, Tailor DV, van Rossum MCW. Unconscious Biases in Neural Populations Coding Multiple Stimuli. Neural Comput 2018; 30:3168-3188. [PMID: 30216141 DOI: 10.1162/neco_a_01130] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Throughout the nervous system, information is commonly coded in activity distributed over populations of neurons. In idealized situations where a single, continuous stimulus is encoded in a homogeneous population code, the value of the encoded stimulus can be read out without bias. However, in many situations, multiple stimuli are simultaneously present; for example, multiple motion patterns might overlap. Here we find that when multiple stimuli that overlap in their neural representation are simultaneously encoded in the population, biases in the read-out emerge. Although the bias disappears in the absence of noise, the bias is remarkably persistent at low noise levels. The bias can be reduced by competitive encoding schemes or by employing complex decoders. To study the origin of the bias, we develop a novel general framework based on gaussian processes that allows an accurate calculation of the estimate distributions of maximum likelihood decoders, and reveals that the distribution of estimates is bimodal for overlapping stimuli. The results have implications for neural coding and behavioral experiments on, for instance, overlapping motion patterns.
Collapse
Affiliation(s)
- Sander W Keemink
- Institute for Adaptive and Neural Computation, School of Informatics, University of Edinburgh, Edinburgh EH8 9AB, U.K., and Bernstein Center Freiburg, Faculty of Biology, University of Freiburg, 79104 Freiburg, Germany
| | - Dharmesh V Tailor
- Institute for Adaptive and Neural Computation, School of Informatics, University of Edinburgh, Edinburgh EH8 9AB, U.K.
| | - Mark C W van Rossum
- School of Psychology and School of Mathematical Sciences, University of Nottingham, Nottingham NH7 2RD, U.K.
| |
Collapse
|
6
|
Bao P, Tsao DY. Representation of multiple objects in macaque category-selective areas. Nat Commun 2018; 9:1774. [PMID: 29720645 PMCID: PMC5932008 DOI: 10.1038/s41467-018-04126-7] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2017] [Accepted: 04/05/2018] [Indexed: 11/13/2022] Open
Abstract
Object recognition in the natural world usually occurs in the presence of multiple surrounding objects, but responses of neurons in inferotemporal (IT) cortex, the large brain area responsible for object recognition, have mostly been studied only to isolated objects. We study rules governing responses to multiple objects by cells in two category-selective regions of macaque IT cortex, the middle lateral face patch (ML) and the middle body patch (MB). We find that responses of single ML and MB cells to pairs of objects can be explained by the widely accepted framework of normalization, with one added ingredient: homogeneous category selectivity of neighboring neurons forming the normalization pool. This rule leads to winner-take-all, contralateral-take-all, or weighted averaging behavior in single cells, depending on the category, spatial configuration, and relative contrast of the two objects. The winner-take-all behavior suggests a potential mechanism for clutter-invariant representation of face and bodies under certain conditions. Inferotemporal cortex (IT) neurons respond to specific objects but the precise neural mechanisms for clutter-invariant representation is not known. Here the authors show that face and body patch IT neurons respond to multiple objects with winner-take-all, contralateral-take-all or weighted averaging depending on the stimulus properties.
Collapse
Affiliation(s)
- Pinglei Bao
- Division of Biology and Biological Engineering, Computation and Neural Systems, California Institute of Technology, Pasadena, CA, 91125, USA.,Howard Hughes Medical Institute, Pasadena, CA, 91125, USA
| | - Doris Y Tsao
- Division of Biology and Biological Engineering, Computation and Neural Systems, California Institute of Technology, Pasadena, CA, 91125, USA. .,Howard Hughes Medical Institute, Pasadena, CA, 91125, USA.
| |
Collapse
|
7
|
Feature-Selective Attention Adaptively Shifts Noise Correlations in Primary Auditory Cortex. J Neurosci 2017; 37:5378-5392. [PMID: 28432139 DOI: 10.1523/jneurosci.3169-16.2017] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2016] [Revised: 02/23/2017] [Accepted: 02/24/2017] [Indexed: 11/21/2022] Open
Abstract
Sensory environments often contain an overwhelming amount of information, with both relevant and irrelevant information competing for neural resources. Feature attention mediates this competition by selecting the sensory features needed to form a coherent percept. How attention affects the activity of populations of neurons to support this process is poorly understood because population coding is typically studied through simulations in which one sensory feature is encoded without competition. Therefore, to study the effects of feature attention on population-based neural coding, investigations must be extended to include stimuli with both relevant and irrelevant features. We measured noise correlations (rnoise) within small neural populations in primary auditory cortex while rhesus macaques performed a novel feature-selective attention task. We found that the effect of feature-selective attention on rnoise depended not only on the population tuning to the attended feature, but also on the tuning to the distractor feature. To attempt to explain how these observed effects might support enhanced perceptual performance, we propose an extension of a simple and influential model in which shifts in rnoise can simultaneously enhance the representation of the attended feature while suppressing the distractor. These findings present a novel mechanism by which attention modulates neural populations to support sensory processing in cluttered environments.SIGNIFICANCE STATEMENT Although feature-selective attention constitutes one of the building blocks of listening in natural environments, its neural bases remain obscure. To address this, we developed a novel auditory feature-selective attention task and measured noise correlations (rnoise) in rhesus macaque A1 during task performance. Unlike previous studies showing that the effect of attention on rnoise depends on population tuning to the attended feature, we show that the effect of attention depends on the tuning to the distractor feature as well. We suggest that these effects represent an efficient process by which sensory cortex simultaneously enhances relevant information and suppresses irrelevant information.
Collapse
|
8
|
Abstract
Zenon Pylyshyn argues that cognitively driven attentional effects do not amount to cognitive penetration of early vision because such effects occur either before or after early vision. Critics object that in fact such effects occur at all levels of perceptual processing. We argue that Pylyshyn’s claim is correct—but not for the reason he emphasizes. Even if his critics are correct that attentional effects are not external to early vision, these effects do not satisfy Pylyshyn’s requirements that the effects be direct and exhibit semantic coherence. In addition, we distinguish our defense from those found in recent work by Raftopoulos and by Firestone and Scholl, argue that attention should not be assimilated to expectation, and discuss alternative characterizations of cognitive penetrability, advocating a kind of pluralism.
Collapse
Affiliation(s)
- Steven Gross
- Department of Philosophy, Johns Hopkins University Baltimore, MD, USA
| |
Collapse
|
9
|
Li K, Kozyrev V, Kyllingsbæk S, Treue S, Ditlevsen S, Bundesen C. Neurons in Primate Visual Cortex Alternate between Responses to Multiple Stimuli in Their Receptive Field. Front Comput Neurosci 2016; 10:141. [PMID: 28082892 PMCID: PMC5187355 DOI: 10.3389/fncom.2016.00141] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2016] [Accepted: 12/12/2016] [Indexed: 11/26/2022] Open
Abstract
A fundamental question concerning representation of the visual world in our brain is how a cortical cell responds when presented with more than a single stimulus. We find supportive evidence that most cells presented with a pair of stimuli respond predominantly to one stimulus at a time, rather than a weighted average response. Traditionally, the firing rate is assumed to be a weighted average of the firing rates to the individual stimuli (response-averaging model) (Bundesen et al., 2005). Here, we also evaluate a probability-mixing model (Bundesen et al., 2005), where neurons temporally multiplex the responses to the individual stimuli. This provides a mechanism by which the representational identity of multiple stimuli in complex visual scenes can be maintained despite the large receptive fields in higher extrastriate visual cortex in primates. We compare the two models through analysis of data from single cells in the middle temporal visual area (MT) of rhesus monkeys when presented with two separate stimuli inside their receptive field with attention directed to one of the two stimuli or outside the receptive field. The spike trains were modeled by stochastic point processes, including memory effects of past spikes and attentional effects, and statistical model selection between the two models was performed by information theoretic measures as well as the predictive accuracy of the models. As an auxiliary measure, we also tested for uni- or multimodality in interspike interval distributions, and performed a correlation analysis of simultaneously recorded pairs of neurons, to evaluate population behavior.
Collapse
Affiliation(s)
- Kang Li
- Department of Mathematical Sciences, University of CopenhagenCopenhagen, Denmark; Department of Psychology, University of CopenhagenCopenhagen, Denmark
| | - Vladislav Kozyrev
- Cognitive Neuroscience Laboratory, German Primate CenterGoettingen, Germany; Bernstein Center for Computational NeuroscienceGoettingen, Germany; Chair Theory of Cognitive Systems, Institute for Neuroinformatics, Ruhr University BochumBochum, Germany; Visual Cognition Lab, Department of Medicine/Physiology, University of FribourgFribourg, Switzerland
| | - Søren Kyllingsbæk
- Department of Psychology, University of Copenhagen Copenhagen, Denmark
| | - Stefan Treue
- Cognitive Neuroscience Laboratory, German Primate CenterGoettingen, Germany; Bernstein Center for Computational NeuroscienceGoettingen, Germany; Faculty for Biology and Psychology, Goettingen UniversityGeottingen, Germany
| | - Susanne Ditlevsen
- Department of Mathematical Sciences, University of Copenhagen Copenhagen, Denmark
| | - Claus Bundesen
- Department of Psychology, University of Copenhagen Copenhagen, Denmark
| |
Collapse
|
10
|
Berberian N, MacPherson A, Giraud E, Richardson L, Thivierge JP. Neuronal pattern separation of motion-relevant input in LIP activity. J Neurophysiol 2016; 117:738-755. [PMID: 27881719 DOI: 10.1152/jn.00145.2016] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2016] [Accepted: 11/18/2016] [Indexed: 11/22/2022] Open
Abstract
In various regions of the brain, neurons discriminate sensory stimuli by decreasing the similarity between ambiguous input patterns. Here, we examine whether this process of pattern separation may drive the rapid discrimination of visual motion stimuli in the lateral intraparietal area (LIP). Starting with a simple mean-rate population model that captures neuronal activity in LIP, we show that overlapping input patterns can be reformatted dynamically to give rise to separated patterns of neuronal activity. The population model predicts that a key ingredient of pattern separation is the presence of heterogeneity in the response of individual units. Furthermore, the model proposes that pattern separation relies on heterogeneity in the temporal dynamics of neural activity and not merely in the mean firing rates of individual neurons over time. We confirm these predictions in recordings of macaque LIP neurons and show that the accuracy of pattern separation is a strong predictor of behavioral performance. Overall, results propose that LIP relies on neuronal pattern separation to facilitate decision-relevant discrimination of sensory stimuli.NEW & NOTEWORTHY A new hypothesis is proposed on the role of the lateral intraparietal (LIP) region of cortex during rapid decision making. This hypothesis suggests that LIP alters the representation of ambiguous inputs to reduce their overlap, thus improving sensory discrimination. A combination of computational modeling, theoretical analysis, and electrophysiological data shows that the pattern separation hypothesis links neural activity to behavior and offers novel predictions on the role of LIP during sensory discrimination.
Collapse
Affiliation(s)
- Nareg Berberian
- Center for Neural Dynamics and School of Psychology, University of Ottawa, Ottawa, Ontario, Canada; and
| | - Amanda MacPherson
- Department of Neuroscience, McGill University, Montréal, Québec, Canada
| | - Eloïse Giraud
- Center for Neural Dynamics and School of Psychology, University of Ottawa, Ottawa, Ontario, Canada; and
| | - Lydia Richardson
- Center for Neural Dynamics and School of Psychology, University of Ottawa, Ottawa, Ontario, Canada; and
| | - J-P Thivierge
- Center for Neural Dynamics and School of Psychology, University of Ottawa, Ottawa, Ontario, Canada; and
| |
Collapse
|
11
|
Serences JT. Neural mechanisms of information storage in visual short-term memory. Vision Res 2016; 128:53-67. [PMID: 27668990 DOI: 10.1016/j.visres.2016.09.010] [Citation(s) in RCA: 118] [Impact Index Per Article: 14.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2016] [Revised: 09/02/2016] [Accepted: 09/21/2016] [Indexed: 11/26/2022]
Abstract
The capacity to briefly memorize fleeting sensory information supports visual search and behavioral interactions with relevant stimuli in the environment. Traditionally, studies investigating the neural basis of visual short term memory (STM) have focused on the role of prefrontal cortex (PFC) in exerting executive control over what information is stored and how it is adaptively used to guide behavior. However, the neural substrates that support the actual storage of content-specific information in STM are more controversial, with some attributing this function to PFC and others to the specialized areas of early visual cortex that initially encode incoming sensory stimuli. In contrast to these traditional views, I will review evidence suggesting that content-specific information can be flexibly maintained in areas across the cortical hierarchy ranging from early visual cortex to PFC. While the factors that determine exactly where content-specific information is represented are not yet entirely clear, recognizing the importance of task-demands and better understanding the operation of non-spiking neural codes may help to constrain new theories about how memories are maintained at different resolutions, across different timescales, and in the presence of distracting information.
Collapse
Affiliation(s)
- John T Serences
- Department of Psychology, Neurosciences Graduate Program, and the Kavli Institute for Mind and Brain, University of California, San Diego, United States.
| |
Collapse
|
12
|
Roth ZN. Functional MRI Representational Similarity Analysis Reveals a Dissociation between Discriminative and Relative Location Information in the Human Visual System. Front Integr Neurosci 2016; 10:16. [PMID: 27242455 PMCID: PMC4876365 DOI: 10.3389/fnint.2016.00016] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2015] [Accepted: 03/14/2016] [Indexed: 11/13/2022] Open
Abstract
Neural responses in visual cortex are governed by a topographic mapping from retinal locations to cortical responses. Moreover, at the voxel population level early visual cortex (EVC) activity enables accurate decoding of stimuli locations. However, in many cases information enabling one to discriminate between locations (i.e., discriminative information) may be less relevant than information regarding the relative location of two objects (i.e., relative information). For example, when planning to grab a cup, determining whether the cup is located at the same retinal location as the hand is hardly relevant, whereas the location of the cup relative to the hand is crucial for performing the action. We have previously used multivariate pattern analysis techniques to measure discriminative location information, and found the highest levels in EVC, in line with other studies. Here we show, using representational similarity analysis, that availability of discriminative information in fMRI activation patterns does not entail availability of relative information. Specifically, we find that relative location information can be reliably extracted from activity patterns in posterior intraparietal sulcus (pIPS), but not from EVC, where we find the spatial representation to be warped. We further show that this variability in relative information levels between regions can be explained by a computational model based on an array of receptive fields. Moreover, when the model's receptive fields are extended to include inhibitory surround regions, the model can account for the spatial warping in EVC. These results demonstrate how size and shape properties of receptive fields in human visual cortex contribute to the transformation of discriminative spatial representations into relative spatial representations along the visual stream.
Collapse
Affiliation(s)
- Zvi N Roth
- The Edmond and Lily Safra Center for Brain Sciences, The Hebrew UniversityJerusalem, Israel; Department of Neurobiology, The Hebrew UniversityJerusalem, Israel
| |
Collapse
|
13
|
Distributed and Dynamic Neural Encoding of Multiple Motion Directions of Transparently Moving Stimuli in Cortical Area MT. J Neurosci 2016; 35:16180-98. [PMID: 26658869 DOI: 10.1523/jneurosci.2175-15.2015] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2023] Open
Abstract
UNLABELLED Segmenting visual scenes into distinct objects and surfaces is a fundamental visual function. To better understand the underlying neural mechanism, we investigated how neurons in the middle temporal cortex (MT) of macaque monkeys represent overlapping random-dot stimuli moving transparently in slightly different directions. It has been shown that the neuronal response elicited by two stimuli approximately follows the average of the responses elicited by the constituent stimulus components presented alone. In this scheme of response pooling, the ability to segment two simultaneously presented motion directions is limited by the width of the tuning curve to motion in a single direction. We found that, although the population-averaged neuronal tuning showed response averaging, subgroups of neurons showed distinct patterns of response tuning and were capable of representing component directions that were separated by a small angle--less than the tuning width to unidirectional stimuli. One group of neurons preferentially represented the component direction at a specific side of the bidirectional stimuli, weighting one stimulus component more strongly than the other. Another group of neurons pooled the component responses nonlinearly and showed two separate peaks in their tuning curves even when the average of the component responses was unimodal. We also show for the first time that the direction tuning of MT neurons evolved from initially representing the vector-averaged direction of slightly different stimuli to gradually representing the component directions. Our results reveal important neural processes underlying image segmentation and suggest that information about slightly different stimulus components is computed dynamically and distributed across neurons. SIGNIFICANCE STATEMENT Natural scenes often contain multiple entities. The ability to segment visual scenes into distinct objects and surfaces is fundamental to sensory processing and is crucial for generating the perception of our environment. Because cortical neurons are broadly tuned to a given visual feature, segmenting two stimuli that differ only slightly is a challenge for the visual system. In this study, we discovered that many neurons in the visual cortex are capable of representing individual components of slightly different stimuli by selectively and nonlinearly pooling the responses elicited by the stimulus components. We also show for the first time that the neural representation of individual stimulus components developed over a period of ∼70-100 ms, revealing a dynamic process of image segmentation.
Collapse
|
14
|
There Is a "U" in Clutter: Evidence for Robust Sparse Codes Underlying Clutter Tolerance in Human Vision. J Neurosci 2016; 35:14148-59. [PMID: 26490856 DOI: 10.1523/jneurosci.1211-15.2015] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/27/2022] Open
Abstract
UNLABELLED The ability to recognize objects in clutter is crucial for human vision, yet the underlying neural computations remain poorly understood. Previous single-unit electrophysiology recordings in inferotemporal cortex in monkeys and fMRI studies of object-selective cortex in humans have shown that the responses to pairs of objects can sometimes be well described as a weighted average of the responses to the constituent objects. Yet, from a computational standpoint, it is not clear how the challenge of object recognition in clutter can be solved if downstream areas must disentangle the identity of an unknown number of individual objects from the confounded average neuronal responses. An alternative idea is that recognition is based on a subpopulation of neurons that are robust to clutter, i.e., that do not show response averaging, but rather robust object-selective responses in the presence of clutter. Here we show that simulations using the HMAX model of object recognition in cortex can fit the aforementioned single-unit and fMRI data, showing that the averaging-like responses can be understood as the result of responses of object-selective neurons to suboptimal stimuli. Moreover, the model shows how object recognition can be achieved by a sparse readout of neurons whose selectivity is robust to clutter. Finally, the model provides a novel prediction about human object recognition performance, namely, that target recognition ability should show a U-shaped dependency on the similarity of simultaneously presented clutter objects. This prediction is confirmed experimentally, supporting a simple, unifying model of how the brain performs object recognition in clutter. SIGNIFICANCE STATEMENT The neural mechanisms underlying object recognition in cluttered scenes (i.e., containing more than one object) remain poorly understood. Studies have suggested that neural responses to multiple objects correspond to an average of the responses to the constituent objects. Yet, it is unclear how the identities of an unknown number of objects could be disentangled from a confounded average response. Here, we use a popular computational biological vision model to show that averaging-like responses can result from responses of clutter-tolerant neurons to suboptimal stimuli. The model also provides a novel prediction, that human detection ability should show a U-shaped dependency on target-clutter similarity, which is confirmed experimentally, supporting a simple, unifying account of how the brain performs object recognition in clutter.
Collapse
|
15
|
Yu Z, Chen F, Dong J, Dai Q. Sampling-based causal inference in cue combination and its neural implementation. Neurocomputing 2016. [DOI: 10.1016/j.neucom.2015.10.045] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
|