1
|
Marvan T, Phillips WA. Cellular mechanisms of cooperative context-sensitive predictive inference. CURRENT RESEARCH IN NEUROBIOLOGY 2024; 6:100129. [PMID: 38665363 PMCID: PMC11043869 DOI: 10.1016/j.crneur.2024.100129] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2023] [Revised: 02/14/2024] [Accepted: 03/25/2024] [Indexed: 04/28/2024] Open
Abstract
We argue that prediction success maximization is a basic objective of cognition and cortex, that it is compatible with but distinct from prediction error minimization, that neither objective requires subtractive coding, that there is clear neurobiological evidence for the amplification of predicted signals, and that we are unconvinced by evidence proposed in support of subtractive coding. We outline recent discoveries showing that pyramidal cells on which our cognitive capabilities depend usually transmit information about input to their basal dendrites and amplify that transmission when input to their distal apical dendrites provides a context that agrees with the feedforward basal input in that both are depolarizing, i.e., both are excitatory rather than inhibitory. Though these intracellular discoveries require a level of technical expertise that is beyond the current abilities of most neuroscience labs, they are not controversial and acclaimed as groundbreaking. We note that this cellular cooperative context-sensitivity greatly enhances the cognitive capabilities of the mammalian neocortex, and that much remains to be discovered concerning its evolution, development, and pathology.
Collapse
Affiliation(s)
- Tomáš Marvan
- Institute of Philosophy, Czech Academy of Sciences (CAS), Czech Republic
| | | |
Collapse
|
2
|
Goris RLT, Coen-Cagli R, Miller KD, Priebe NJ, Lengyel M. Response sub-additivity and variability quenching in visual cortex. Nat Rev Neurosci 2024; 25:237-252. [PMID: 38374462 DOI: 10.1038/s41583-024-00795-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/24/2024] [Indexed: 02/21/2024]
Abstract
Sub-additivity and variability are ubiquitous response motifs in the primary visual cortex (V1). Response sub-additivity enables the construction of useful interpretations of the visual environment, whereas response variability indicates the factors that limit the precision with which the brain can do this. There is increasing evidence that experimental manipulations that elicit response sub-additivity often also quench response variability. Here, we provide an overview of these phenomena and suggest that they may have common origins. We discuss empirical findings and recent model-based insights into the functional operations, computational objectives and circuit mechanisms underlying V1 activity. These different modelling approaches all predict that response sub-additivity and variability quenching often co-occur. The phenomenology of these two response motifs, as well as many of the insights obtained about them in V1, generalize to other cortical areas. Thus, the connection between response sub-additivity and variability quenching may be a canonical motif across the cortex.
Collapse
Affiliation(s)
- Robbe L T Goris
- Center for Perceptual Systems, University of Texas at Austin, Austin, TX, USA.
| | - Ruben Coen-Cagli
- Department of Systems and Computational Biology, Albert Einstein College of Medicine, Bronx, NY, USA
- Dominick P. Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY, USA
- Department of Ophthalmology and Visual Sciences, Albert Einstein College of Medicine, Bronx, NY, USA
| | - Kenneth D Miller
- Center for Theoretical Neuroscience, Columbia University, New York, NY, USA
- Kavli Institute for Brain Science, Columbia University, New York, NY, USA
- Dept. of Neuroscience, College of Physicians and Surgeons, Columbia University, New York, NY, USA
- Morton B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
- Swartz Program in Theoretical Neuroscience, Columbia University, New York, NY, USA
| | - Nicholas J Priebe
- Center for Learning and Memory, University of Texas at Austin, Austin, TX, USA
| | - Máté Lengyel
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK
- Center for Cognitive Computation, Department of Cognitive Science, Central European University, Budapest, Hungary
| |
Collapse
|
3
|
Goldstein A, Grinstein-Dabush A, Schain M, Wang H, Hong Z, Aubrey B, Schain M, Nastase SA, Zada Z, Ham E, Feder A, Gazula H, Buchnik E, Doyle W, Devore S, Dugan P, Reichart R, Friedman D, Brenner M, Hassidim A, Devinsky O, Flinker A, Hasson U. Alignment of brain embeddings and artificial contextual embeddings in natural language points to common geometric patterns. Nat Commun 2024; 15:2768. [PMID: 38553456 PMCID: PMC10980748 DOI: 10.1038/s41467-024-46631-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2022] [Accepted: 03/04/2024] [Indexed: 04/02/2024] Open
Abstract
Contextual embeddings, derived from deep language models (DLMs), provide a continuous vectorial representation of language. This embedding space differs fundamentally from the symbolic representations posited by traditional psycholinguistics. We hypothesize that language areas in the human brain, similar to DLMs, rely on a continuous embedding space to represent language. To test this hypothesis, we densely record the neural activity patterns in the inferior frontal gyrus (IFG) of three participants using dense intracranial arrays while they listened to a 30-minute podcast. From these fine-grained spatiotemporal neural recordings, we derive a continuous vectorial representation for each word (i.e., a brain embedding) in each patient. Using stringent zero-shot mapping we demonstrate that brain embeddings in the IFG and the DLM contextual embedding space have common geometric patterns. The common geometric patterns allow us to predict the brain embedding in IFG of a given left-out word based solely on its geometrical relationship to other non-overlapping words in the podcast. Furthermore, we show that contextual embeddings capture the geometry of IFG embeddings better than static word embeddings. The continuous brain embedding space exposes a vector-based neural code for natural language processing in the human brain.
Collapse
Affiliation(s)
- Ariel Goldstein
- Business School, Data Science department and Cognitive Department, Hebrew University, Jerusalem, Israel.
- Google Research, Tel Aviv, Israel.
| | | | | | - Haocheng Wang
- Department of Psychology and the Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Zhuoqiao Hong
- Department of Psychology and the Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Bobbi Aubrey
- Department of Psychology and the Neuroscience Institute, Princeton University, Princeton, NJ, USA
- New York University Grossman School of Medicine, New York, NY, USA
| | | | - Samuel A Nastase
- Department of Psychology and the Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Zaid Zada
- Department of Psychology and the Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Eric Ham
- Department of Psychology and the Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | | | - Harshvardhan Gazula
- Department of Psychology and the Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | | | - Werner Doyle
- New York University Grossman School of Medicine, New York, NY, USA
| | - Sasha Devore
- New York University Grossman School of Medicine, New York, NY, USA
| | - Patricia Dugan
- New York University Grossman School of Medicine, New York, NY, USA
| | - Roi Reichart
- Faculty of Industrial Engineering and Management, Technion, Israel Institute of Technology, Haifa, Israel
| | - Daniel Friedman
- New York University Grossman School of Medicine, New York, NY, USA
| | - Michael Brenner
- Google Research, Tel Aviv, Israel
- School of Engineering and Applied Science, Harvard University, Cambridge, MA, USA
| | | | - Orrin Devinsky
- New York University Grossman School of Medicine, New York, NY, USA
| | - Adeen Flinker
- New York University Grossman School of Medicine, New York, NY, USA
- New York University Tandon School of Engineering, Brooklyn, NY, USA
| | - Uri Hasson
- Google Research, Tel Aviv, Israel
- Department of Psychology and the Neuroscience Institute, Princeton University, Princeton, NJ, USA
| |
Collapse
|
4
|
Tring E, Dipoppa M, Ringach DL. On the contrast response function of adapted neural populations. J Neurophysiol 2024; 131:446-453. [PMID: 38264786 DOI: 10.1152/jn.00413.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Revised: 01/02/2024] [Accepted: 01/22/2024] [Indexed: 01/25/2024] Open
Abstract
The magnitude of neural responses in sensory cortex depends on the intensity of a stimulus and its probability of being observed within the environment. How these two variables combine to influence the overall response of cortical populations remains unknown. Here we show that, in primary visual cortex, the vector magnitude of the population response is described by a separable power law that factors the intensity of a stimulus and its probability. Moreover, the discriminability between two contrast levels in a cortical population is proportional to the logarithm of the contrast ratio.NEW & NOTEWORTHY The magnitude of neural responses in sensory cortex depends on the intensity of a stimulus and its probability of being observed within the environment. The authors show that, in primary visual cortex, the vector magnitude of the population response is described by a separable power law that factors the intensity of a stimulus and its probability.
Collapse
Affiliation(s)
- Elaine Tring
- Department of Neurobiology, David Geffen School of Medicine, University of California, Los Angeles, California, United States
| | - Mario Dipoppa
- Department of Neurobiology, David Geffen School of Medicine, University of California, Los Angeles, California, United States
| | - Dario L Ringach
- Department of Psychology, David Geffen School of Medicine, University of California, Los Angeles, California, United States
- Department of Neurobiology, David Geffen School of Medicine, University of California, Los Angeles, California, United States
| |
Collapse
|
5
|
Weiss O, Bounds HA, Adesnik H, Coen-Cagli R. Modeling the diverse effects of divisive normalization on noise correlations. PLoS Comput Biol 2023; 19:e1011667. [PMID: 38033166 PMCID: PMC10715670 DOI: 10.1371/journal.pcbi.1011667] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Revised: 12/12/2023] [Accepted: 11/07/2023] [Indexed: 12/02/2023] Open
Abstract
Divisive normalization, a prominent descriptive model of neural activity, is employed by theories of neural coding across many different brain areas. Yet, the relationship between normalization and the statistics of neural responses beyond single neurons remains largely unexplored. Here we focus on noise correlations, a widely studied pairwise statistic, because its stimulus and state dependence plays a central role in neural coding. Existing models of covariability typically ignore normalization despite empirical evidence suggesting it affects correlation structure in neural populations. We therefore propose a pairwise stochastic divisive normalization model that accounts for the effects of normalization and other factors on covariability. We first show that normalization modulates noise correlations in qualitatively different ways depending on whether normalization is shared between neurons, and we discuss how to infer when normalization signals are shared. We then apply our model to calcium imaging data from mouse primary visual cortex (V1), and find that it accurately fits the data, often outperforming a popular alternative model of correlations. Our analysis indicates that normalization signals are often shared between V1 neurons in this dataset. Our model will enable quantifying the relation between normalization and covariability in a broad range of neural systems, which could provide new constraints on circuit mechanisms of normalization and their role in information transmission and representation.
Collapse
Affiliation(s)
- Oren Weiss
- Department of Systems and Computational Biology, Albert Einstein College of Medicine, Bronx, New York, United States of America
| | - Hayley A. Bounds
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, California, United States of America
| | - Hillel Adesnik
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, California, United States of America
- Department of Molecular and Cell Biology, University of California, Berkeley, Berkeley, California, United States of America
| | - Ruben Coen-Cagli
- Department of Systems and Computational Biology, Albert Einstein College of Medicine, Bronx, New York, United States of America
- Dominick P. Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, New York, United States of America
- Department of Ophthalmology and Visual Sciences, Albert Einstein College of Medicine, Bronx, New York, United States of America
| |
Collapse
|
6
|
Bulatov A, Marma V, Bulatova N, Loginovič J, Vaitiekaitis G. Effects of normalized summation in the visual illusion of extent. Atten Percept Psychophys 2023; 85:2422-2436. [PMID: 37369970 DOI: 10.3758/s13414-023-02744-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/05/2023] [Indexed: 06/29/2023]
Abstract
In the present study, the features of summation of effects caused by contextual distracting dots in the length-matching task (a variant of the filled-space illusion) were investigated. In the first two series of psychophysical experiments, the illusion magnitude was measured as a function of the displacement of distractors (either single or double sets of dots) orthogonally to the main axis of the stimulus. It was demonstrated that with increasing displacement, the illusion smoothly decreases for a single set of distractors, while for two sets, the illusion first increases to a certain maximum value, and then gradually decreases. In the third and fourth series of experiments, magnitude of the illusion was measured as a function of the luminance of one set of distracting dots, while the luminance of the other set was fixed. It has been shown that increasing the luminance until the same value is reached for both sets leads to a monotonous growth in the illusion magnitude; after that, the illusion asymptotically decreases to an almost constant level. The theoretical interpretation of the established functional dependencies was performed using a quantitative model based on the assumption that the illusion may arise due to the weighted summation of the distractor-induced normalized neural activity, which leads to the perceptual mislocalization of terminators of stimulus spatial intervals.
Collapse
Affiliation(s)
- Aleksandr Bulatov
- Laboratory of Visual Neurophysiology, Lithuanian University of Health Sciences, Mickevičiaus 9, LT-44307, Kaunas, Lithuania.
- Institute of Biological Systems and Genetics Research, Lithuanian University of Health Sciences, Kaunas, Lithuania.
| | - Vilius Marma
- Laboratory of Visual Neurophysiology, Lithuanian University of Health Sciences, Mickevičiaus 9, LT-44307, Kaunas, Lithuania
- Institute of Biological Systems and Genetics Research, Lithuanian University of Health Sciences, Kaunas, Lithuania
| | - Natalija Bulatova
- Institute of Biological Systems and Genetics Research, Lithuanian University of Health Sciences, Kaunas, Lithuania
| | - Jelena Loginovič
- Institute of Biological Systems and Genetics Research, Lithuanian University of Health Sciences, Kaunas, Lithuania
| | - Gintautas Vaitiekaitis
- Physics, Mathematics, and Biophysics Department, Lithuanian University of Health Sciences, Kaunas, Lithuania
| |
Collapse
|
7
|
Schmid D, Jarvers C, Neumann H. Canonical circuit computations for computer vision. BIOLOGICAL CYBERNETICS 2023; 117:299-329. [PMID: 37306782 PMCID: PMC10600314 DOI: 10.1007/s00422-023-00966-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/30/2022] [Accepted: 05/18/2023] [Indexed: 06/13/2023]
Abstract
Advanced computer vision mechanisms have been inspired by neuroscientific findings. However, with the focus on improving benchmark achievements, technical solutions have been shaped by application and engineering constraints. This includes the training of neural networks which led to the development of feature detectors optimally suited to the application domain. However, the limitations of such approaches motivate the need to identify computational principles, or motifs, in biological vision that can enable further foundational advances in machine vision. We propose to utilize structural and functional principles of neural systems that have been largely overlooked. They potentially provide new inspirations for computer vision mechanisms and models. Recurrent feedforward, lateral, and feedback interactions characterize general principles underlying processing in mammals. We derive a formal specification of core computational motifs that utilize these principles. These are combined to define model mechanisms for visual shape and motion processing. We demonstrate how such a framework can be adopted to run on neuromorphic brain-inspired hardware platforms and can be extended to automatically adapt to environment statistics. We argue that the identified principles and their formalization inspires sophisticated computational mechanisms with improved explanatory scope. These and other elaborated, biologically inspired models can be employed to design computer vision solutions for different tasks and they can be used to advance neural network architectures of learning.
Collapse
Affiliation(s)
- Daniel Schmid
- Institute for Neural Information Processing, Ulm University, James-Franck-Ring, Ulm, 89081 Germany
| | - Christian Jarvers
- Institute for Neural Information Processing, Ulm University, James-Franck-Ring, Ulm, 89081 Germany
| | - Heiko Neumann
- Institute for Neural Information Processing, Ulm University, James-Franck-Ring, Ulm, 89081 Germany
| |
Collapse
|
8
|
Graham NV, Wolfson SS. Varying test-pattern duration to explore the dynamics of contrast-comparison and contrast-normalization processes. J Vis 2023; 23:15. [PMID: 36689217 PMCID: PMC9896861 DOI: 10.1167/jov.23.1.15] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/24/2023] Open
Abstract
In this paper, we examine the dynamics of contrast-comparison and contrast-normalization processes. Observers adapted (for 1 second) to a grid of Gabor patches at one contrast; then a test pattern (which varied in duration from 12 ms to 3012 ms) was shown; and then the adapt pattern was shown again (1 second). All the Gabor patches in all the adapt patterns had 50% contrast. The test pattern was the same as the adapt pattern except that the Gabor patches in the test pattern had two different contrasts; the test contrasts varied from row to row (horizontal test pattern) or column to column (vertical test pattern). The task was to identify the orientation of the contrast variation in the test pattern (in other words, the observer performed a second-order orientation identification task). The two contrasts in each test pattern were varied while keeping the difference between the two contrasts constant. We have previously found that the observer's performance is poor for test patterns containing contrasts both above and below the adapt patterns' contrast (what we have called the "straddle effect") when the test duration is approximately 100 ms. Here, we find the straddle effect persists at all test durations we used. Other features of the results varied dramatically with test duration. We find that a simple model containing contrast-comparison and contrast-normalization processes provides a good explanation for the psychophysical results. The results provide some insight into the dynamics of these processes.
Collapse
Affiliation(s)
- Norma V. Graham
- Department of Psychology, Columbia University, New York, NY, USA,
| | | |
Collapse
|
9
|
Jun NY, Ruff DA, Kramer LE, Bowes B, Tokdar ST, Cohen MR, Groh JM. Coordinated multiplexing of information about separate objects in visual cortex. eLife 2022; 11:e76452. [PMID: 36444983 PMCID: PMC9708082 DOI: 10.7554/elife.76452] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2021] [Accepted: 10/21/2022] [Indexed: 11/30/2022] Open
Abstract
Sensory receptive fields are large enough that they can contain more than one perceptible stimulus. How, then, can the brain encode information about each of the stimuli that may be present at a given moment? We recently showed that when more than one stimulus is present, single neurons can fluctuate between coding one vs. the other(s) across some time period, suggesting a form of neural multiplexing of different stimuli (Caruso et al., 2018). Here, we investigate (a) whether such coding fluctuations occur in early visual cortical areas; (b) how coding fluctuations are coordinated across the neural population; and (c) how coordinated coding fluctuations depend on the parsing of stimuli into separate vs. fused objects. We found coding fluctuations do occur in macaque V1 but only when the two stimuli form separate objects. Such separate objects evoked a novel pattern of V1 spike count ('noise') correlations involving distinct distributions of positive and negative values. This bimodal correlation pattern was most pronounced among pairs of neurons showing the strongest evidence for coding fluctuations or multiplexing. Whether a given pair of neurons exhibited positive or negative correlations depended on whether the two neurons both responded better to the same object or had different object preferences. Distinct distributions of spike count correlations based on stimulus preferences were also seen in V4 for separate objects but not when two stimuli fused to form one object. These findings suggest multiple objects evoke different response dynamics than those evoked by single stimuli, lending support to the multiplexing hypothesis and suggesting a means by which information about multiple objects can be preserved despite the apparent coarseness of sensory coding.
Collapse
Affiliation(s)
- Na Young Jun
- Department of Neurobiology, Duke UniversityDurhamUnited States
- Center for Cognitive Neuroscience, Duke UniversityDurhamUnited States
- Duke Institute for Brain SciencesDurhamUnited States
| | - Douglas A Ruff
- Department of Neuroscience, University of PittsburghPittsburghUnited States
- Center for the Neural Basis of Cognition, University of PittsburghPittsburghUnited States
| | - Lily E Kramer
- Department of Neuroscience, University of PittsburghPittsburghUnited States
- Center for the Neural Basis of Cognition, University of PittsburghPittsburghUnited States
| | - Brittany Bowes
- Department of Neuroscience, University of PittsburghPittsburghUnited States
- Center for the Neural Basis of Cognition, University of PittsburghPittsburghUnited States
| | - Surya T Tokdar
- Department of Statistical Science, Duke UniversityDurhamUnited States
| | - Marlene R Cohen
- Department of Neuroscience, University of PittsburghPittsburghUnited States
- Center for the Neural Basis of Cognition, University of PittsburghPittsburghUnited States
| | - Jennifer M Groh
- Department of Neurobiology, Duke UniversityDurhamUnited States
- Center for Cognitive Neuroscience, Duke UniversityDurhamUnited States
- Duke Institute for Brain SciencesDurhamUnited States
- Department of Psychology and Neuroscience, Duke UniversityDurhamUnited States
- Department of Biomedical Engineering, Duke UniversityDurhamUnited States
- Department of Computer Science, Duke UniversityDurhamUnited States
| |
Collapse
|
10
|
In vivo extracellular recordings of thalamic and cortical visual responses reveal V1 connectivity rules. Proc Natl Acad Sci U S A 2022; 119:e2207032119. [PMID: 36191204 PMCID: PMC9564935 DOI: 10.1073/pnas.2207032119] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/01/2023] Open
Abstract
The brain's connectome provides the scaffold for canonical neural computations. However, a comparison of connectivity studies in the mouse primary visual cortex (V1) reveals that the average number and strength of connections between specific neuron types can vary. Can variability in V1 connectivity measurements coexist with canonical neural computations? We developed a theory-driven approach to deduce V1 network connectivity from visual responses in mouse V1 and visual thalamus (dLGN). Our method revealed that the same recorded visual responses were captured by multiple connectivity configurations. Remarkably, the magnitude and selectivity of connectivity weights followed a specific order across most of the inferred connectivity configurations. We argue that this order stems from the specific shapes of the recorded contrast response functions and contrast invariance of orientation tuning. Remarkably, despite variability across connectivity studies, connectivity weights computed from individual published connectivity reports followed the order we identified with our method, suggesting that the relations between the weights, rather than their magnitudes, represent a connectivity motif supporting canonical V1 computations.
Collapse
|
11
|
Louie K. Asymmetric and adaptive reward coding via normalized reinforcement learning. PLoS Comput Biol 2022; 18:e1010350. [PMID: 35862443 PMCID: PMC9345478 DOI: 10.1371/journal.pcbi.1010350] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2022] [Revised: 08/02/2022] [Accepted: 07/01/2022] [Indexed: 11/18/2022] Open
Abstract
Learning is widely modeled in psychology, neuroscience, and computer science by prediction error-guided reinforcement learning (RL) algorithms. While standard RL assumes linear reward functions, reward-related neural activity is a saturating, nonlinear function of reward; however, the computational and behavioral implications of nonlinear RL are unknown. Here, we show that nonlinear RL incorporating the canonical divisive normalization computation introduces an intrinsic and tunable asymmetry in prediction error coding. At the behavioral level, this asymmetry explains empirical variability in risk preferences typically attributed to asymmetric learning rates. At the neural level, diversity in asymmetries provides a computational mechanism for recently proposed theories of distributional RL, allowing the brain to learn the full probability distribution of future rewards. This behavioral and computational flexibility argues for an incorporation of biologically valid value functions in computational models of learning and decision-making. Reinforcement learning models are widely used to characterize reward-driven learning in biological and computational agents. Standard reinforcement learning models use linear value functions, despite strong empirical evidence that biological value representations are nonlinear functions of external rewards. Here, we examine the properties of a biologically-based nonlinear reinforcement learning algorithm employing the canonical divisive normalization function, a neural computation commonly found in sensory, cognitive, and reward coding. We show that this normalized reinforcement learning algorithm implements a simple but powerful control of how reward learning reflects relative gains and losses. This property explains diverse behavioral and neural phenomena, and suggests the importance of using biologically valid value functions in computational models of learning and decision-making.
Collapse
Affiliation(s)
- Kenway Louie
- Center for Neural Science, New York University, New York, United States of America
- Neuroscience Institute, New York University Grossman School of Medicine, New York, United States of America
- * E-mail:
| |
Collapse
|
12
|
Shirhatti V, Ravishankar P, Ray S. Gamma oscillations in primate primary visual cortex are severely attenuated by small stimulus discontinuities. PLoS Biol 2022; 20:e3001666. [PMID: 35700175 PMCID: PMC9197048 DOI: 10.1371/journal.pbio.3001666] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2022] [Accepted: 05/10/2022] [Indexed: 11/22/2022] Open
Abstract
Gamma oscillations (30 to 80 Hz) have been hypothesized to play an important role in feature binding, based on the observation that continuous long bars induce stronger gamma in the visual cortex than bars with a small gap. Recently, many studies have shown that natural images, which have discontinuities in several low-level features, do not induce strong gamma oscillations, questioning their role in feature binding. However, the effect of different discontinuities on gamma has not been well studied. To address this, we recorded spikes and local field potential from 2 monkeys while they were shown gratings with discontinuities in 4 attributes: space, orientation, phase, or contrast. We found that while these discontinuities only had a modest effect on spiking activity, gamma power drastically reduced in all cases, suggesting that gamma could be a resonant phenomenon. An excitatory–inhibitory population model with stimulus-tuned recurrent inputs showed such resonant properties. Therefore, gamma could be a signature of excitation–inhibition balance, which gets disrupted due to discontinuities. Gamma oscillations (30-80 Hz) in visual cortex have been hypothesized to play an important role in feature binding, but this role has recently been questioned. This study shows that visual stimulus-induced gamma oscillations are highly attenuated with even small discontinuities in the stimulus. This "resonant" behaviour can be explained by a simple excitatory-inhibitory model in which discontinuities lead to a small reduction in lateral inputs.
Collapse
Affiliation(s)
- Vinay Shirhatti
- Centre for Neuroscience, Indian Institute of Science, Bengaluru, India
- IISc Mathematics Initiative, Indian Institute of Science, Bengaluru, India
| | | | - Supratim Ray
- Centre for Neuroscience, Indian Institute of Science, Bengaluru, India
- IISc Mathematics Initiative, Indian Institute of Science, Bengaluru, India
- * E-mail:
| |
Collapse
|
13
|
Denison RN, Carrasco M, Heeger DJ. A dynamic normalization model of temporal attention. Nat Hum Behav 2021; 5:1674-1685. [PMID: 34140658 PMCID: PMC8678377 DOI: 10.1038/s41562-021-01129-1] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2020] [Accepted: 04/29/2021] [Indexed: 02/05/2023]
Abstract
Vision is dynamic, handling a continuously changing stream of input, yet most models of visual attention are static. Here, we develop a dynamic normalization model of visual temporal attention and constrain it with new psychophysical human data. We manipulated temporal attention-the prioritization of visual information at specific points in time-to a sequence of two stimuli separated by a variable time interval. Voluntary temporal attention improved perceptual sensitivity only over a specific interval range. To explain these data, we modelled voluntary and involuntary attentional gain dynamics. Voluntary gain enhancement took the form of a limited resource over short time intervals, which recovered over time. Taken together, our theoretical and experimental results formalize and generalize the idea of limited attentional resources across space at a single moment to limited resources across time at a single location.
Collapse
Affiliation(s)
- Rachel N Denison
- Department of Psychology and Center for Neural Science, New York University, New York, NY, USA.
| | - Marisa Carrasco
- Department of Psychology and Center for Neural Science, New York University, New York, NY, USA
| | - David J Heeger
- Department of Psychology and Center for Neural Science, New York University, New York, NY, USA
| |
Collapse
|
14
|
Divisive normalization unifies disparate response signatures throughout the human visual hierarchy. Proc Natl Acad Sci U S A 2021; 118:2108713118. [PMID: 34772812 PMCID: PMC8609633 DOI: 10.1073/pnas.2108713118] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/30/2021] [Indexed: 01/04/2023] Open
Abstract
A canonical neural computation is a mathematical operation applied by the brain in a wide variety of contexts and capable of explaining and unifying seemingly unrelated neural and perceptual phenomena. Here, we use a combination of state-of-the-art experiments (ultra-high-field functional MRI) and mathematical methods (population receptive field [pRF] modeling) to uniquely demonstrate the role of divisive normalization (DN) as the canonical neural computation underlying visuospatial responses throughout the human visual hierarchy. The DN pRF model provides a tool to investigate and interpret the computational processes underlying neural responses in human and animal recordings, but also in clinical and cognitive dimensions. Neural processing is hypothesized to apply the same mathematical operations in a variety of contexts, implementing so-called canonical neural computations. Divisive normalization (DN) is considered a prime candidate for a canonical computation. Here, we propose a population receptive field (pRF) model based on DN and evaluate it using ultra-high-field functional MRI (fMRI). The DN model parsimoniously captures seemingly disparate response signatures with a single computation, superseding existing pRF models in both performance and biological plausibility. We observe systematic variations in specific DN model parameters across the visual hierarchy and show how they relate to differences in response modulation and visuospatial information integration. The DN model delivers a unifying framework for visuospatial responses throughout the human visual hierarchy and provides insights into its underlying information-encoding computations. These findings extend the role of DN as a canonical computation to neuronal populations throughout the human visual hierarchy.
Collapse
|
15
|
Ernst UA, Chen X, Bohnenkamp L, Galashan FO, Wegener D. Dynamic divisive normalization circuits explain and predict change detection in monkey area MT. PLoS Comput Biol 2021; 17:e1009595. [PMID: 34767547 PMCID: PMC8612546 DOI: 10.1371/journal.pcbi.1009595] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2020] [Revised: 11/24/2021] [Accepted: 10/27/2021] [Indexed: 11/24/2022] Open
Abstract
Sudden changes in visual scenes often indicate important events for behavior. For their quick and reliable detection, the brain must be capable to process these changes as independently as possible from its current activation state. In motion-selective area MT, neurons respond to instantaneous speed changes with pronounced transients, often far exceeding the expected response as derived from their speed tuning profile. We here show that this complex, non-linear behavior emerges from the combined temporal dynamics of excitation and divisive inhibition, and provide a comprehensive mathematical analysis. A central prediction derived from this investigation is that attention increases the steepness of the transient response irrespective of the activation state prior to a stimulus change, and irrespective of the sign of the change (i.e. irrespective of whether the stimulus is accelerating or decelerating). Extracellular recordings of attention-dependent representation of both speed increments and decrements confirmed this prediction and suggest that improved change detection derives from basic computations in a canonical cortical circuitry.
Collapse
Affiliation(s)
- Udo A. Ernst
- Computational Neurophysics Lab, Institute for Theoretical Physics, University of Bremen, Bremen, Germany
| | - Xiao Chen
- Computational Neurophysics Lab, Institute for Theoretical Physics, University of Bremen, Bremen, Germany
| | - Lisa Bohnenkamp
- Computational Neurophysics Lab, Institute for Theoretical Physics, University of Bremen, Bremen, Germany
| | | | - Detlef Wegener
- Brain Research Institute, University of Bremen, Bremen, Germany
| |
Collapse
|
16
|
Festa D, Aschner A, Davila A, Kohn A, Coen-Cagli R. Neuronal variability reflects probabilistic inference tuned to natural image statistics. Nat Commun 2021; 12:3635. [PMID: 34131142 PMCID: PMC8206154 DOI: 10.1038/s41467-021-23838-x] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2020] [Accepted: 05/19/2021] [Indexed: 11/23/2022] Open
Abstract
Neuronal activity in sensory cortex fluctuates over time and across repetitions of the same input. This variability is often considered detrimental to neural coding. The theory of neural sampling proposes instead that variability encodes the uncertainty of perceptual inferences. In primary visual cortex (V1), modulation of variability by sensory and non-sensory factors supports this view. However, it is unknown whether V1 variability reflects the statistical structure of visual inputs, as would be required for inferences correctly tuned to the statistics of the natural environment. Here we combine analysis of image statistics and recordings in macaque V1 to show that probabilistic inference tuned to natural image statistics explains the widely observed dependence between spike count variance and mean, and the modulation of V1 activity and variability by spatial context in images. Our results show that the properties of a basic aspect of cortical responses-their variability-can be explained by a probabilistic representation tuned to naturalistic inputs.
Collapse
Affiliation(s)
- Dylan Festa
- Department of Systems and Computational Biology, Albert Einstein College of Medicine, Bronx, NY, USA
| | - Amir Aschner
- Dominick Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY, USA
| | - Aida Davila
- Dominick Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY, USA
| | - Adam Kohn
- Department of Systems and Computational Biology, Albert Einstein College of Medicine, Bronx, NY, USA
- Dominick Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY, USA
- Department of Ophthalmology and Visual Sciences, Albert Einstein College of Medicine, Bronx, NY, USA
| | - Ruben Coen-Cagli
- Department of Systems and Computational Biology, Albert Einstein College of Medicine, Bronx, NY, USA.
- Dominick Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY, USA.
| |
Collapse
|