1
|
Di Tullio RW, Wei L, Balasubramanian V. Slow and steady: auditory features for discriminating animal vocalizations. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.06.20.599962. [PMID: 39005308 PMCID: PMC11244870 DOI: 10.1101/2024.06.20.599962] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/16/2024]
Abstract
We propose that listeners can use temporal regularities - spectro-temporal correlations that change smoothly over time - to discriminate animal vocalizations within and between species. To test this idea, we used Slow Feature Analysis (SFA) to find the most temporally regular components of vocalizations from birds (blue jay, house finch, American yellow warbler, and great blue heron), humans (English speakers), and rhesus macaques. We projected vocalizations into the learned feature space and tested intra-class (same speaker/species) and inter-class (different speakers/species) auditory discrimination by a trained classifier. We found that: 1) Vocalization discrimination was excellent (> 95%) in all cases; 2) Performance depended primarily on the ~10 most temporally regular features; 3) Most vocalizations are dominated by ~10 features with high temporal regularity; and 4) These regular features are highly correlated with the most predictable components of animal sounds.
Collapse
Affiliation(s)
- Ronald W Di Tullio
- David Rittenhouse Laboratory, Department of Physics and Astronomy, University of Pennsylvania, USA
- Computational Neuroscience Initiative, University of Pennsylvania, USA
| | - Linran Wei
- David Rittenhouse Laboratory, Department of Physics and Astronomy, University of Pennsylkvania, USA
| | - Vijay Balasubramanian
- David Rittenhouse Laboratory, Department of Physics and Astronomy, University of Pennsylvania, USA
- Computational Neuroscience Initiative, University of Pennsylvania, USA
- Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501, USA
| |
Collapse
|
2
|
Matteucci G, Piasini E, Zoccolan D. Unsupervised learning of mid-level visual representations. Curr Opin Neurobiol 2024; 84:102834. [PMID: 38154417 DOI: 10.1016/j.conb.2023.102834] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2023] [Revised: 12/03/2023] [Accepted: 12/05/2023] [Indexed: 12/30/2023]
Abstract
Recently, a confluence between trends in neuroscience and machine learning has brought a renewed focus on unsupervised learning, where sensory processing systems learn to exploit the statistical structure of their inputs in the absence of explicit training targets or rewards. Sophisticated experimental approaches have enabled the investigation of the influence of sensory experience on neural self-organization and its synaptic bases. Meanwhile, novel algorithms for unsupervised and self-supervised learning have become increasingly popular both as inspiration for theories of the brain, particularly for the function of intermediate visual cortical areas, and as building blocks of real-world learning machines. Here we review some of these recent developments, placing them in historical context and highlighting some research lines that promise exciting breakthroughs in the near future.
Collapse
Affiliation(s)
- Giulio Matteucci
- Department of Basic Neurosciences, University of Geneva, Geneva, 1206, Switzerland. https://twitter.com/giulio_matt
| | - Eugenio Piasini
- International School for Advanced Studies (SISSA), Trieste, 34136, Italy
| | - Davide Zoccolan
- International School for Advanced Studies (SISSA), Trieste, 34136, Italy.
| |
Collapse
|
3
|
Victor JD, Rizvi SM, Bush JW, Conte MM. Discrimination of textures with spatial correlations and multiple gray levels. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2023; 40:237-258. [PMID: 36821194 PMCID: PMC9971653 DOI: 10.1364/josaa.472553] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/12/2022] [Accepted: 12/05/2022] [Indexed: 06/18/2023]
Abstract
Analysis of visual texture is important for many key steps in early vision. We study visual sensitivity to image statistics in three families of textures that include multiple gray levels and correlations in two spatial dimensions. Sensitivities to positive and negative correlations are approximately independent of correlation sign, and signals from different kinds of correlations combine quadratically. We build a computational model, fully constrained by prior studies of sensitivity to uncorrelated textures and black-and-white textures with spatial correlations. The model accounts for many features of the new data, including sign-independence, quadratic combination, and the dependence on gray-level distribution.
Collapse
Affiliation(s)
- Jonathan D. Victor
- Feil Family Brain and Mind Research Institute, Weill Cornell Medical College, 1300 York Avenue, New York, NY 10065, USA
| | - Syed M. Rizvi
- Feil Family Brain and Mind Research Institute, Weill Cornell Medical College, 1300 York Avenue, New York, NY 10065, USA
- Currently with Centerlight Healthcare, 136-65 37th Ave., Flushing, NY 11354, USA
| | - Jacob W. Bush
- Feil Family Brain and Mind Research Institute, Weill Cornell Medical College, 1300 York Avenue, New York, NY 10065, USA
- Currently with Shopify, 151 O’Connor St Ground floor, Ottawa, ON K2P 2L8, Canada
| | - Mary M. Conte
- Feil Family Brain and Mind Research Institute, Weill Cornell Medical College, 1300 York Avenue, New York, NY 10065, USA
| |
Collapse
|
4
|
Lieber JD, Lee GM, Majaj NJ, Movshon JA. Sensitivity to naturalistic texture relies primarily on high spatial frequencies. J Vis 2023; 23:4. [PMID: 36745452 PMCID: PMC9910384 DOI: 10.1167/jov.23.2.4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2022] [Accepted: 11/19/2022] [Indexed: 02/07/2023] Open
Abstract
Natural images contain information at multiple spatial scales. Though we understand how early visual mechanisms split multiscale images into distinct spatial frequency channels, we do not know how the outputs of these channels are processed further by mid-level visual mechanisms. We have recently developed a texture discrimination task that uses synthetic, multi-scale, "naturalistic" textures to isolate these mid-level mechanisms. Here, we use three experimental manipulations (image blur, image rescaling, and eccentric viewing) to show that perceptual sensitivity to naturalistic structure is strongly dependent on features at high object spatial frequencies (measured in cycles/image). As a result, sensitivity depends on a texture acuity limit, a property of the visual system that sets the highest retinal spatial frequency (measured in cycles/degree) at which observers can detect naturalistic features. Analysis of the texture images using a model observer analysis shows that naturalistic image features at high object spatial frequencies carry more task-relevant information than those at low object spatial frequencies. That is, the dependence of sensitivity on high object spatial frequencies is a property of the texture images, rather than a property of the visual system. Accordingly, we find human observers' ability to extract naturalistic information (their efficiency) is similar for all object spatial frequencies. We conclude that the mid-level mechanisms that underlie perceptual sensitivity effectively extract information from all image features below the texture acuity limit, regardless of their retinal and object spatial frequency.
Collapse
Affiliation(s)
- Justin D Lieber
- Center for Neural Science, New York University, New York, NY, USA
| | - Gerick M Lee
- Center for Neural Science, New York University, New York, NY, USA
| | - Najib J Majaj
- Center for Neural Science, New York University, New York, NY, USA
| | | |
Collapse
|
5
|
Buccellato A, Zang D, Zilio F, Gomez-Pilar J, Wang Z, Qi Z, Zheng R, Xu Z, Wu X, Bisiacchi P, Del Felice A, Mao Y, Northoff G. Disrupted relationship between intrinsic neural timescales and alpha peak frequency during unconscious states - A high-density EEG study. Neuroimage 2023; 265:119802. [PMID: 36503159 DOI: 10.1016/j.neuroimage.2022.119802] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2022] [Revised: 11/22/2022] [Accepted: 12/07/2022] [Indexed: 12/13/2022] Open
Abstract
Our brain processes the different timescales of our environment's temporal input stochastics. Is such a temporal input processing mechanism key for consciousness? To address this research question, we calculated measures of input processing on shorter (alpha peak frequency, APF) and longer (autocorrelation window, ACW) timescales on resting-state high-density EEG (256 channels) recordings and compared them across different consciousness levels (awake/conscious, ketamine and sevoflurane anaesthesia, unresponsive wakefulness, minimally conscious state). We replicate and extend previous findings of: (i) significantly longer ACW values, consistently over all states of unconsciousness, as measured with ACW-0 (an unprecedented longer version of the well-know ACW-50); (ii) significantly slower APF values, as measured with frequency sliding, in all four unconscious states. Most importantly, we report a highly significant correlation of ACW-0 and APF in the conscious state, while their relationship is disrupted in the unconscious states. In sum, we demonstrate the relevance of the brain's capacity for input processing on shorter (APF) and longer (ACW) timescales - including their relationship - for consciousness. Albeit indirectly, e.g., through the analysis of electrophysiological activity at rest, this supports the mechanism of temporo-spatial alignment to the environment's temporal input stochastics, through relating different neural timescales, as one key predisposing factor of consciousness.
Collapse
Affiliation(s)
- Andrea Buccellato
- Padova Neuroscience Center, University of Padova, Padova, Italy; Department of General Psychology, University of Padova, Padova, Italy.
| | - Di Zang
- Department of Neurosurgery, Huashan Hospital, Shanghai Medical College, Fudan University,Shanghai, 200040, China; Shanghai Key Laboratory of Brain Function and Restoration and Neural Regeneration, Shanghai, 200040, China; State Key Laboratory of Medical Neurobiology, MOE Frontiers Center for Brain Science, School of Basic Medical Sciences and Institutes of Brain Science, Fudan University, Shanghai, 200032, China; National Center for Neurological Disorders, Shanghai, 200040, China; Neurosurgical Institute of Fudan University, Shanghai, 200040, China; Shanghai Clinical Medical Center of Neurosurgery, Shanghai, 200040, China
| | - Federico Zilio
- Department of Philosophy, Sociology, Education and Applied Psychology, University of Padova, Padua, Italy
| | - Javier Gomez-Pilar
- Biomedical Engineering Group, University of Valladolid, Paseo de Belén, 15, Valladolid 47011, Spain; Centro de Investigación Biomédica en Red en Bioingeniería, Biomateriales y Nanomedicina, (CIBER-BBN), Valladolid, Spain
| | - Zhe Wang
- Department of Neurosurgery, Huashan Hospital, Shanghai Medical College, Fudan University,Shanghai, 200040, China; Shanghai Key Laboratory of Brain Function and Restoration and Neural Regeneration, Shanghai, 200040, China; State Key Laboratory of Medical Neurobiology, MOE Frontiers Center for Brain Science, School of Basic Medical Sciences and Institutes of Brain Science, Fudan University, Shanghai, 200032, China; National Center for Neurological Disorders, Shanghai, 200040, China; Neurosurgical Institute of Fudan University, Shanghai, 200040, China; Shanghai Clinical Medical Center of Neurosurgery, Shanghai, 200040, China
| | - Zengxin Qi
- Department of Neurosurgery, Huashan Hospital, Shanghai Medical College, Fudan University,Shanghai, 200040, China; Shanghai Key Laboratory of Brain Function and Restoration and Neural Regeneration, Shanghai, 200040, China; State Key Laboratory of Medical Neurobiology, MOE Frontiers Center for Brain Science, School of Basic Medical Sciences and Institutes of Brain Science, Fudan University, Shanghai, 200032, China; National Center for Neurological Disorders, Shanghai, 200040, China; Neurosurgical Institute of Fudan University, Shanghai, 200040, China; Shanghai Clinical Medical Center of Neurosurgery, Shanghai, 200040, China
| | - Ruizhe Zheng
- Department of Neurosurgery, Huashan Hospital, Shanghai Medical College, Fudan University,Shanghai, 200040, China; Shanghai Key Laboratory of Brain Function and Restoration and Neural Regeneration, Shanghai, 200040, China; State Key Laboratory of Medical Neurobiology, MOE Frontiers Center for Brain Science, School of Basic Medical Sciences and Institutes of Brain Science, Fudan University, Shanghai, 200032, China; National Center for Neurological Disorders, Shanghai, 200040, China; Neurosurgical Institute of Fudan University, Shanghai, 200040, China; Shanghai Clinical Medical Center of Neurosurgery, Shanghai, 200040, China
| | - Zeyu Xu
- Department of Neurosurgery, Huashan Hospital, Shanghai Medical College, Fudan University,Shanghai, 200040, China; Shanghai Key Laboratory of Brain Function and Restoration and Neural Regeneration, Shanghai, 200040, China; State Key Laboratory of Medical Neurobiology, MOE Frontiers Center for Brain Science, School of Basic Medical Sciences and Institutes of Brain Science, Fudan University, Shanghai, 200032, China; National Center for Neurological Disorders, Shanghai, 200040, China; Neurosurgical Institute of Fudan University, Shanghai, 200040, China; Shanghai Clinical Medical Center of Neurosurgery, Shanghai, 200040, China
| | - Xuehai Wu
- Department of Neurosurgery, Huashan Hospital, Shanghai Medical College, Fudan University,Shanghai, 200040, China; Shanghai Key Laboratory of Brain Function and Restoration and Neural Regeneration, Shanghai, 200040, China; State Key Laboratory of Medical Neurobiology, MOE Frontiers Center for Brain Science, School of Basic Medical Sciences and Institutes of Brain Science, Fudan University, Shanghai, 200032, China; National Center for Neurological Disorders, Shanghai, 200040, China; Neurosurgical Institute of Fudan University, Shanghai, 200040, China; Shanghai Clinical Medical Center of Neurosurgery, Shanghai, 200040, China
| | - Patrizia Bisiacchi
- Padova Neuroscience Center, University of Padova, Padova, Italy; Department of General Psychology, University of Padova, Padova, Italy
| | - Alessandra Del Felice
- Padova Neuroscience Center, University of Padova, Padova, Italy; Department of Neuroscience, Section of Rehabilitation, University of Padova, Padova, Italy
| | - Ying Mao
- Department of Neurosurgery, Huashan Hospital, Shanghai Medical College, Fudan University,Shanghai, 200040, China; Shanghai Key Laboratory of Brain Function and Restoration and Neural Regeneration, Shanghai, 200040, China; State Key Laboratory of Medical Neurobiology, MOE Frontiers Center for Brain Science, School of Basic Medical Sciences and Institutes of Brain Science, Fudan University, Shanghai, 200032, China; National Center for Neurological Disorders, Shanghai, 200040, China; Neurosurgical Institute of Fudan University, Shanghai, 200040, China; Shanghai Clinical Medical Center of Neurosurgery, Shanghai, 200040, China.
| | - Georg Northoff
- Mind, Brain Imaging and Neuroethics Research Unit, Institute of Mental Health, Royal Ottawa Mental Health Centre and University of Ottawa, Ottawa, Ontario K1Z7K4, Canada; Mental Health Center, Zhejiang University School of Medicine, Hangzhou 310013, Zhejiang Province, China; Centre for Cognition and Brain Disorders, Hangzhou Normal University, Hangzhou 310013, Zhejiang Province, China.
| |
Collapse
|
6
|
Tesileanu T, Piasini E, Balasubramanian V. Efficient processing of natural scenes in visual cortex. Front Cell Neurosci 2022; 16:1006703. [PMID: 36545653 PMCID: PMC9760692 DOI: 10.3389/fncel.2022.1006703] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2022] [Accepted: 11/17/2022] [Indexed: 12/12/2022] Open
Abstract
Neural circuits in the periphery of the visual, auditory, and olfactory systems are believed to use limited resources efficiently to represent sensory information by adapting to the statistical structure of the natural environment. This "efficient coding" principle has been used to explain many aspects of early visual circuits including the distribution of photoreceptors, the mosaic geometry and center-surround structure of retinal receptive fields, the excess OFF pathways relative to ON pathways, saccade statistics, and the structure of simple cell receptive fields in V1. We know less about the extent to which such adaptations may occur in deeper areas of cortex beyond V1. We thus review recent developments showing that the perception of visual textures, which depends on processing in V2 and beyond in mammals, is adapted in rats and humans to the multi-point statistics of luminance in natural scenes. These results suggest that central circuits in the visual brain are adapted for seeing key aspects of natural scenes. We conclude by discussing how adaptation to natural temporal statistics may aid in learning and representing visual objects, and propose two challenges for the future: (1) explaining the distribution of shape sensitivity in the ventral visual stream from the statistics of object shape in natural images, and (2) explaining cell types of the vertebrate retina in terms of feature detectors that are adapted to the spatio-temporal structures of natural stimuli. We also discuss how new methods based on machine learning may complement the normative, principles-based approach to theoretical neuroscience.
Collapse
Affiliation(s)
- Tiberiu Tesileanu
- Center for Computational Neuroscience, Flatiron Institute, New York, NY, United States
| | - Eugenio Piasini
- Scuola Internazionale Superiore di Studi Avanzati (SISSA), Trieste, Italy
| | - Vijay Balasubramanian
- Department of Physics and Astronomy, David Rittenhouse Laboratory, University of Pennsylvania, Philadelphia, PA, United States
- Santa Fe Institute, Santa Fe, NM, United States
| |
Collapse
|
7
|
Guidolin A, Desroches M, Victor JD, Purpura KP, Rodrigues S. Geometry of spiking patterns in early visual cortex: a topological data analytic approach. J R Soc Interface 2022; 19:20220677. [PMID: 36382589 PMCID: PMC9667368 DOI: 10.1098/rsif.2022.0677] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2022] [Accepted: 10/21/2022] [Indexed: 11/17/2022] Open
Abstract
In the brain, spiking patterns live in a high-dimensional space of neurons and time. Thus, determining the intrinsic structure of this space presents a theoretical and experimental challenge. To address this challenge, we introduce a new framework for applying topological data analysis (TDA) to spike train data and use it to determine the geometry of spiking patterns in the visual cortex. Key to our approach is a parametrized family of distances based on the timing of spikes that quantifies the dissimilarity between neuronal responses. We applied TDA to visually driven single-unit and multiple single-unit spiking activity in macaque V1 and V2. TDA across timescales reveals a common geometry for spiking patterns in V1 and V2 which, among simple models, is most similar to that of a low-dimensional space endowed with Euclidean or hyperbolic geometry with modest curvature. Remarkably, the inferred geometry depends on timescale and is clearest for the timescales that are important for encoding contrast, orientation and spatial correlations.
Collapse
Affiliation(s)
- Andrea Guidolin
- MCEN Team, BCAM – Basque Center for Applied Mathematics, 48009 Bilbao, Basque Country, Spain
- Department of Mathematics, KTH Royal Institute of Technology, SE-100 44 Stockholm, Sweden
| | - Mathieu Desroches
- MathNeuro Team, Inria at Université Côte d’Azur, 06902 Sophia Antipolis, France
| | - Jonathan D. Victor
- Feil Family Brain and Mind Research Institute, Weill Cornell Medical College, New York, NY 10065, USA
| | - Keith P. Purpura
- Feil Family Brain and Mind Research Institute, Weill Cornell Medical College, New York, NY 10065, USA
| | - Serafim Rodrigues
- MCEN Team, BCAM – Basque Center for Applied Mathematics, 48009 Bilbao, Basque Country, Spain
- Ikerbasque – The Basque Foundation for Science, 48009 Bilbao, Basque Country, Spain
| |
Collapse
|
8
|
Bauer M. How does an organism extract relevant information from transcription factor concentrations? Biochem Soc Trans 2022; 50:1365-1376. [PMID: 36111776 PMCID: PMC9704516 DOI: 10.1042/bst20220333] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2022] [Revised: 08/16/2022] [Accepted: 08/17/2022] [Indexed: 09/10/2024]
Abstract
How does an organism regulate its genes? The involved regulation typically occurs in terms of a signal processing chain: an externally applied stimulus or a maternally supplied transcription factor leads to the expression of some downstream genes, which, in turn, are transcription factors for further genes. Especially during development, these transcription factors are frequently expressed in amounts where noise is still important; yet, the signals that they provide must not be lost in the noise. Thus, the organism needs to extract exactly relevant information in the signal. New experimental approaches involving single-molecule measurements at high temporal precision as well as increased precision in manipulations directly on the genome are allowing us to tackle this question anew. These new experimental advances mean that also from the theoretical side, theoretical advances should be possible. In this review, I will describe, specifically on the example of fly embryo gene regulation, how theoretical approaches, especially from inference and information theory, can help in understanding gene regulation. To do so, I will first review some more traditional theoretical models for gene regulation, followed by a brief discussion of information-theoretical approaches and when they can be applied. I will then introduce early fly development as an exemplary system where such information-theoretical approaches have traditionally been applied and can be applied; I will specifically focus on how one such method, namely the information bottleneck approach, has recently been used to infer structural features of enhancer architecture.
Collapse
Affiliation(s)
- Marianne Bauer
- Bionanoscience Department, Delft University of Technology, van der Maasweg 9, 2629 Delft, The Netherlands
- Joseph Henry Laboratories of Physics, Princeton University, Princeton, NJ 08544, U.S.A
- Lewis–Sigler Institute for Integrative Genomics Princeton University, Princeton, NJ 08544, U.S.A
| |
Collapse
|
9
|
Seenivasan P, Narayanan R. Efficient information coding and degeneracy in the nervous system. Curr Opin Neurobiol 2022; 76:102620. [PMID: 35985074 PMCID: PMC7613645 DOI: 10.1016/j.conb.2022.102620] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2022] [Revised: 07/01/2022] [Accepted: 07/07/2022] [Indexed: 11/25/2022]
Abstract
Efficient information coding (EIC) is a universal biological framework rooted in the fundamental principle that system responses should match their natural stimulus statistics for maximizing environmental information. Quantitatively assessed through information theory, such adaptation to the environment occurs at all biological levels and timescales. The context dependence of environmental stimuli and the need for stable adaptations make EIC a daunting task. We argue that biological complexity is the principal architect that subserves deft execution of stable EIC. Complexity in a system is characterized by several functionally segregated subsystems that show a high degree of functional integration when they interact with each other. Complex biological systems manifest heterogeneities and degeneracy, wherein structurally different subsystems could interact to yield the same functional outcome. We argue that complex systems offer several choices that effectively implement EIC and homeostasis for each of the different contexts encountered by the system.
Collapse
Affiliation(s)
- Pavithraa Seenivasan
- Cellular Neurophysiology Laboratory, Molecular Biophysics Unit, Indian Institute of Science, Bangalore, 560012, India. https://twitter.com/PaveeSeeni
| | - Rishikesh Narayanan
- Cellular Neurophysiology Laboratory, Molecular Biophysics Unit, Indian Institute of Science, Bangalore, 560012, India.
| |
Collapse
|
10
|
Functional recursion of orientation cues in figure-ground separation. Vision Res 2022; 197:108047. [PMID: 35691090 PMCID: PMC9262819 DOI: 10.1016/j.visres.2022.108047] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2021] [Revised: 03/16/2022] [Accepted: 03/23/2022] [Indexed: 11/23/2022]
Abstract
Visual texture is an important cue to figure-ground organization. While processing of texture differences is a prerequisite for the use of this cue to extract figure-ground organization, these stages are distinct processes. One potential indicator of this distinction is the possibility that texture statistics play a different role in the figure vs. in the ground. To determine whether this is the case, we probed figure-ground processing with a family of local image statistics that specified textures that varied in the strength and spatial scale of structure, and the extent to which features are oriented. For image statistics that generated approximately isotropic textures, the threshold for identification of figure-ground structure was determined by the difference in correlation strength in figure vs. ground, independent of whether the correlations were present in figure, ground, or both. However, for image statistics with strong orientation content, thresholds were up to two times higher for correlations in the ground, vs. the figure. This held equally for texture-defined objects with convex or concave boundaries, indicating that these threshold differences are driven by border ownership, not boundary shape. Similar threshold differences were found for presentation times ranging from 125 to 500 ms. These findings identify a qualitative difference in how texture is used for figure-ground analysis, vs. texture discrimination. Additionally, it reveals a functional recursion: texture differences are needed to identify tentative boundaries and consequent scene organization into figure and ground, but then scene organization modifies sensitivity to texture differences according to the figure-ground assignment.
Collapse
|
11
|
Caramellino R, Piasini E, Buccellato A, Carboncino A, Balasubramanian V, Zoccolan D. Rat sensitivity to multipoint statistics is predicted by efficient coding of natural scenes. eLife 2021; 10:e72081. [PMID: 34872633 PMCID: PMC8651284 DOI: 10.7554/elife.72081] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2021] [Accepted: 11/18/2021] [Indexed: 01/23/2023] Open
Abstract
Efficient processing of sensory data requires adapting the neuronal encoding strategy to the statistics of natural stimuli. Previously, in Hermundstad et al., 2014, we showed that local multipoint correlation patterns that are most variable in natural images are also the most perceptually salient for human observers, in a way that is compatible with the efficient coding principle. Understanding the neuronal mechanisms underlying such adaptation to image statistics will require performing invasive experiments that are impossible in humans. Therefore, it is important to understand whether a similar phenomenon can be detected in animal species that allow for powerful experimental manipulations, such as rodents. Here we selected four image statistics (from single- to four-point correlations) and trained four groups of rats to discriminate between white noise patterns and binary textures containing variable intensity levels of one of such statistics. We interpreted the resulting psychometric data with an ideal observer model, finding a sharp decrease in sensitivity from two- to four-point correlations and a further decrease from four- to three-point. This ranking fully reproduces the trend we previously observed in humans, thus extending a direct demonstration of efficient coding to a species where neuronal and developmental processes can be interrogated and causally manipulated.
Collapse
Affiliation(s)
| | - Eugenio Piasini
- Computational Neuroscience Initiative, University of PennsylvaniaPhiladelphiaUnited States
| | - Andrea Buccellato
- Visual Neuroscience Lab, International School for Advanced StudiesTriesteItaly
| | - Anna Carboncino
- Visual Neuroscience Lab, International School for Advanced StudiesTriesteItaly
| | - Vijay Balasubramanian
- Computational Neuroscience Initiative, University of PennsylvaniaPhiladelphiaUnited States
| | - Davide Zoccolan
- Visual Neuroscience Lab, International School for Advanced StudiesTriesteItaly
| |
Collapse
|
12
|
DiTullio RW, Balasubramanian V. Dynamical self-organization and efficient representation of space by grid cells. Curr Opin Neurobiol 2021; 70:206-213. [PMID: 34861597 PMCID: PMC8688296 DOI: 10.1016/j.conb.2021.11.007] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2021] [Accepted: 11/09/2021] [Indexed: 10/19/2022]
Abstract
To plan trajectories and navigate, animals must maintain a mental representation of the environment and their own position within it. This "cognitive map" is thought to be supported in part by the entorhinal cortex, where grid cells are active when an animal occupies the vertices of a scaling hierarchy of periodic lattices of locations in an enclosure. Here, we review computational developments which suggest that the grid cell network is: (a) efficient, providing required spatial resolution with a minimum number of neurons, (b) self-organizing, dynamically coordinating the structure and scale of the responses, and (c) adaptive, re-organizing in response to changes in landmarks and the structure of the boundaries of spaces. We consider these ideas in light of recent discoveries of similar structures in the mental representation of abstract spaces of shapes and smells, and in other brain areas, and highlight promising directions for future research.
Collapse
Affiliation(s)
- Ronald W. DiTullio
- David Rittenhouse Laboratories & Computational Neuroscience Initiative, University of Pennsylvania, Philadelphia, PA 19104
| | - Vijay Balasubramanian
- David Rittenhouse Laboratories & Computational Neuroscience Initiative, University of Pennsylvania, Philadelphia, PA 19104
| |
Collapse
|
13
|
Redundancy between spectral and higher-order texture statistics for natural image segmentation. Vision Res 2021; 187:55-65. [PMID: 34217005 DOI: 10.1016/j.visres.2021.06.007] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2020] [Revised: 06/09/2021] [Accepted: 06/11/2021] [Indexed: 11/23/2022]
Abstract
Visual texture, defined by local image statistics, provides important information to the human visual system for perceptual segmentation. Second-order or spectral statistics (equivalent to the Fourier power spectrum) are a well-studied segmentation cue. However, the role of higher-order statistics (HOS) in segmentation remains unclear, particularly for natural images. Recent experiments indicate that, in peripheral vision, the HOS of the widely adopted Portilla-Simoncelli texture model are a weak segmentation cue compared to spectral statistics, despite the fact that both are necessary to explain other perceptual phenomena and to support high-quality texture synthesis. Here we test whether this discrepancy reflects a property of natural image statistics. First, we observe that differences in spectral statistics across segments of natural images are redundant with differences in HOS. Second, using linear and nonlinear classifiers, we show that each set of statistics individually affords high performance in natural scenes and texture segmentation tasks, but combining spectral statistics and HOS produces relatively small improvements. Third, we find that HOS improve segmentation for a subset of images, although these images are difficult to identify. We also find that different subsets of HOS improve segmentation to a different extent, in agreement with previous physiological and perceptual work. These results show that the HOS add modestly to spectral statistics for natural image segmentation. We speculate that tuning to natural image statistics under resource constraints could explain the weak contribution of HOS to perceptual segmentation in human peripheral vision.
Collapse
|