Number detectors spontaneously emerge in a deep neural network designed for visual object recognition

High-level visual processing in the lateral geniculate nucleus revealed using goal-driven deep learning

BACKGROUND

The Lateral Geniculate Nucleus (LGN) is an essential contributor to high-level visual processing despite being an early subcortical area in the visual system. Current LGN computational models focus on its basic properties, with less emphasis on its role in high-level vision.

NEW METHOD

We propose a high-level approach for encoding mouse LGN neural responses to natural scenes. This approach employs two deep neural networks (DNNs); namely VGG16 and ResNet50, as goal-driven models. We use these models as tools to better understand visual features encoded in the LGN.

RESULTS

Early layers of the DNNs represent the best LGN models. We also demonstrate that numerosity, as a high-level visual feature, is encoded, along with other visual features, in LGN neural activity. Results demonstrate that intermediate layers are better in representing numerosity compared to early layers. Early layers are better at predicting simple visual features, while intermediate layers are better at predicting more complex features. Finally, we show that an ensemble model of an early and an intermediate layer achieves high neural prediction accuracy and numerosity representation.

COMPARISON WITH EXISTING METHOD(S)

Our approach emphasizes the role of analyzing the inner workings of DNNs to demonstrate the representation of a high-level feature such as numerosity in the LGN, as opposed to the common belief about the simplicity of the LGN.

CONCLUSIONS

We demonstrate that goal-driven DNNs can be used as high-level vision models of the LGN for neural prediction and as an exploration tool to better understand the role of the LGN.

Machine Learning Techniques for Simulating Human Psychophysical Testing of Low-Resolution Phosphene Face Images in Artificial Vision

To evaluate the quality of artificial visual percepts generated by emerging methodologies, researchers often rely on labor-intensive and tedious human psychophysical experiments. These experiments necessitate repeated iterations upon any major/minor modifications in the hardware/software configurations. Here, the capacity of standard machine learning (ML) models is investigated to accurately replicate quaternary match-to-sample tasks using low-resolution facial images represented by arrays of phosphenes as input stimuli. Initially, the performance of the ML models trained to approximate innate human facial recognition abilities across a dataset comprising 3600 phosphene images of human faces is analyzed. Subsequently, due to the time constraints and the potential for subject fatigue, the psychophysical test is limited to presenting only 720 low-resolution phosphene images to 36 human subjects. Notably, the superior model adeptly mirrors the behavioral trend of human subjects, offering precise predictions for 8 out of 9 phosphene quality levels on the overlapping test queries. Subsequently, human recognition performances for untested phosphene images are predicted, streamlining the process and minimizing the need for additional psychophysical tests. The findings underscore the transformative potential of ML in reshaping the research paradigm of visual prosthetics, facilitating the expedited advancement of prostheses.

Is there an innate sense of number in the brain?

The approximate number system or «sense of number» is a crucial, presymbolic mechanism enabling animals to estimate quantities, which is essential for survival in various contexts (eg estimating numerosities of social companions, prey, predators, and so on). Behavioral studies indicate that a sense of number is widespread across vertebrates and invertebrates. Specific brain regions such as the intraparietal sulcus and prefrontal cortex in primates, or equivalent areas in birds and fish, are involved in numerical estimation, and their activity is modulated by the ratio of quantities. Data gathered across species strongly suggest similar evolutionary pressures for number estimation pointing to a likely common origin, at least across vertebrates. On the other hand, few studies have investigated the origins of the sense of number. Recent findings, however, have shown that numerosity-selective neurons exist in newborn animals, such as domestic chicks and zebrafish, supporting the hypothesis of an innateness of the approximate number system. Control-rearing experiments on visually naïve animals further support the notion that the sense of number is innate and does not need any specific instructive experience in order to be triggered.

The calculating brain

The human brain possesses neural networks and mechanisms enabling the representation of numbers, basic arithmetic operations, and mathematical reasoning. Without the ability to represent numerical quantity and perform calculations, our scientifically and technically advanced culture would not exist. However, the origins of numerical abilities are grounded in an intuitive understanding of quantity deeply rooted in biology. Nevertheless, more advanced symbolic arithmetic skills require a cultural background with formal mathematical education. In the past two decades, cognitive neuroscience has seen significant progress in understanding the workings of the calculating brain through various methods and model systems. This review begins by exploring the mental and neuronal representations of nonsymbolic numerical quantity and then progresses to symbolic representations acquired in childhood. During arithmetic operations (addition, subtraction, multiplication, and division), these representations are processed and transformed according to arithmetic rules and principles, leveraging different mental strategies and types of arithmetic knowledge that can be dissociated in the brain. Although it was once believed that number processing and calculation originated from the language faculty, it is now evident that mathematical and linguistic abilities are primarily processed independently in the brain. Understanding how the healthy brain processes numerical information is crucial for gaining insights into debilitating numerical disorders, including acquired conditions like acalculia and learning-related calculation disorders such as developmental dyscalculia.

Zero-shot counting with a dual-stream neural network model

To understand a visual scene, observers need to both recognize objects and encode relational structure. For example, a scene comprising three apples requires the observer to encode concepts of "apple" and "three." In the primate brain, these functions rely on dual (ventral and dorsal) processing streams. Object recognition in primates has been successfully modeled with deep neural networks, but how scene structure (including numerosity) is encoded remains poorly understood. Here, we built a deep learning model, based on the dual-stream architecture of the primate brain, which is able to count items "zero-shot"-even if the objects themselves are unfamiliar. Our dual-stream network forms spatial response fields and lognormal number codes that resemble those observed in the macaque posterior parietal cortex. The dual-stream network also makes successful predictions about human counting behavior. Our results provide evidence for an enactive theory of the role of the posterior parietal cortex in visual scene understanding.

Estimating the distribution of numerosity and non-numerical visual magnitudes in natural scenes using computer vision

Humans share with many animal species the ability to perceive and approximately represent the number of objects in visual scenes. This ability improves throughout childhood, suggesting that learning and development play a key role in shaping our number sense. This hypothesis is further supported by computational investigations based on deep learning, which have shown that numerosity perception can spontaneously emerge in neural networks that learn the statistical structure of images with a varying number of items. However, neural network models are usually trained using synthetic datasets that might not faithfully reflect the statistical structure of natural environments, and there is also growing interest in using more ecological visual stimuli to investigate numerosity perception in humans. In this work, we exploit recent advances in computer vision algorithms to design and implement an original pipeline that can be used to estimate the distribution of numerosity and non-numerical magnitudes in large-scale datasets containing thousands of real images depicting objects in daily life situations. We show that in natural visual scenes the frequency of appearance of different numerosities follows a power law distribution. Moreover, we show that the correlational structure for numerosity and continuous magnitudes is stable across datasets and scene types (homogeneous vs. heterogeneous object sets). We suggest that considering such "ecological" pattern of covariance is important to understand the influence of non-numerical visual cues on numerosity judgements.

Evaluating large language models in theory of mind tasks

Eleven large language models (LLMs) were assessed using 40 bespoke false-belief tasks, considered a gold standard in testing theory of mind (ToM) in humans. Each task included a false-belief scenario, three closely matched true-belief control scenarios, and the reversed versions of all four. An LLM had to solve all eight scenarios to solve a single task. Older models solved no tasks; Generative Pre-trained Transformer (GPT)-3-davinci-003 (from November 2022) and ChatGPT-3.5-turbo (from March 2023) solved 20% of the tasks; ChatGPT-4 (from June 2023) solved 75% of the tasks, matching the performance of 6-y-old children observed in past studies. We explore the potential interpretation of these results, including the intriguing possibility that ToM-like ability, previously considered unique to humans, may have emerged as an unintended by-product of LLMs' improving language skills. Regardless of how we interpret these outcomes, they signify the advent of more powerful and socially skilled AI-with profound positive and negative implications.

The divisive normalization model of visual number sense: model predictions and experimental confirmation

Our intuitive sense of number allows rapid estimation for the number of objects (numerosity) in a scene. How does the continuous nature of neural information processing create a discrete representation of number? A neurocomputational model with divisive normalization explains this process and existing data; however, a successful model should not only explain existing data but also generate novel predictions. Here, we experimentally test novel predictions of this model to evaluate its merit for explaining mechanisms of numerosity perception. We did so by consideration of the coherence illusion: the underestimation of number for arrays containing heterogeneous compared to homogeneous items. First, we established the existence of the coherence illusion for homogeneity manipulations of both area and orientation of items in an array. Second, despite the behavioral similarity, the divisive normalization model predicted that these two illusions should reflect activity in different stages of visual processing. Finally, visual evoked potentials from an electroencephalography experiment confirmed these predictions, showing that area and orientation coherence modulate brain responses at distinct latencies and topographies. These results demonstrate the utility of the divisive normalization model for explaining numerosity perception, according to which numerosity perception is a byproduct of canonical neurocomputations that exist throughout the visual pathway.

Shared functional specialization in transformer-based language models and the human brain

When processing language, the brain is thought to deploy specialized computations to construct meaning from complex linguistic structures. Recently, artificial neural networks based on the Transformer architecture have revolutionized the field of natural language processing. Transformers integrate contextual information across words via structured circuit computations. Prior work has focused on the internal representations ("embeddings") generated by these circuits. In this paper, we instead analyze the circuit computations directly: we deconstruct these computations into the functionally-specialized "transformations" that integrate contextual information across words. Using functional MRI data acquired while participants listened to naturalistic stories, we first verify that the transformations account for considerable variance in brain activity across the cortical language network. We then demonstrate that the emergent computations performed by individual, functionally-specialized "attention heads" differentially predict brain activity in specific cortical regions. These heads fall along gradients corresponding to different layers and context lengths in a low-dimensional cortical space.

Digital twins for understanding mechanisms of learning disabilities: Personalized deep neural networks reveal impact of neuronal hyperexcitability

Learning disabilities affect a significant proportion of children worldwide, with far-reaching consequences for their academic, professional, and personal lives. Here we develop digital twins - biologically plausible personalized Deep Neural Networks (pDNNs) - to investigate the neurophysiological mechanisms underlying learning disabilities in children. Our pDNN reproduces behavioral and neural activity patterns observed in affected children, including lower performance accuracy, slower learning rates, neural hyper-excitability, and reduced neural differentiation of numerical problems. Crucially, pDNN models reveal aberrancies in the geometry of manifold structure, providing a comprehensive view of how neural excitability influences both learning performance and the internal structure of neural representations. Our findings not only advance knowledge of the neurophysiological underpinnings of learning differences but also open avenues for targeted, personalized strategies designed to bridge cognitive gaps in affected children. This work reveals the power of digital twins integrating AI and neuroscience to uncover mechanisms underlying neurodevelopmental disorders.

A number sense as an emergent property of the manipulating brain

The ability to understand and manipulate numbers and quantities emerges during childhood, but the mechanism through which humans acquire and develop this ability is still poorly understood. We explore this question through a model, assuming that the learner is able to pick up and place small objects from, and to, locations of its choosing, and will spontaneously engage in such undirected manipulation. We further assume that the learner's visual system will monitor the changing arrangements of objects in the scene and will learn to predict the effects of each action by comparing perception with a supervisory signal from the motor system. We model perception using standard deep networks for feature extraction and classification. Our main finding is that, from learning the task of action prediction, an unexpected image representation emerges exhibiting regularities that foreshadow the perception and representation of numbers and quantity. These include distinct categories for zero and the first few natural numbers, a strict ordering of the numbers, and a one-dimensional signal that correlates with numerical quantity. As a result, our model acquires the ability to estimate numerosity, i.e. the number of objects in the scene, as well as subitization, i.e. the ability to recognize at a glance the exact number of objects in small scenes. Remarkably, subitization and numerosity estimation extrapolate to scenes containing many objects, far beyond the three objects used during training. We conclude that important aspects of a facility with numbers and quantities may be learned with supervision from a simple pre-training task. Our observations suggest that cross-modal learning is a powerful learning mechanism that may be harnessed in artificial intelligence.

Emergence of Emotion Selectivity in Deep Neural Networks Trained to Recognize Visual Objects

Recent neuroimaging studies have shown that the visual cortex plays an important role in representing the affective significance of visual input. The origin of these affect-specific visual representations is debated: they are intrinsic to the visual system versus they arise through reentry from frontal emotion processing structures such as the amygdala. We examined this problem by combining convolutional neural network (CNN) models of the human ventral visual cortex pre-trained on ImageNet with two datasets of affective images. Our results show that (1) in all layers of the CNN models, there were artificial neurons that responded consistently and selectively to neutral, pleasant, or unpleasant images and (2) lesioning these neurons by setting their output to 0 or enhancing these neurons by increasing their gain led to decreased or increased emotion recognition performance respectively. These results support the idea that the visual system may have the intrinsic ability to represent the affective significance of visual input and suggest that CNNs offer a fruitful platform for testing neuroscientific theories.

Emergence of Emotion Selectivity in Deep Neural Networks Trained to Recognize Visual Objects

Recent neuroimaging studies have shown that the visual cortex plays an important role in representing the affective significance of visual input. The origin of these affect-specific visual representations is debated: they are intrinsic to the visual system versus they arise through reentry from frontal emotion processing structures such as the amygdala. We examined this problem by combining convolutional neural network (CNN) models of the human ventral visual cortex pre-trained on ImageNet with two datasets of affective images. Our results show that in all layers of the CNN models, there were artificial neurons that responded consistently and selectively to neutral, pleasant, or unpleasant images and lesioning these neurons by setting their output to zero or enhancing these neurons by increasing their gain led to decreased or increased emotion recognition performance respectively. These results support the idea that the visual system may have the intrinsic ability to represent the affective significance of visual input and suggest that CNNs offer a fruitful platform for testing neuroscientific theories.

Disparate processing of numerosity and associated continuous magnitudes in rats

The studies of number sense in different species are severely hampered by the inevitable entanglement of non-numerical attributes inherent in nonsymbolic stimuli representing numerosity, resulting in contrasting theories of numerosity processing. Here, we developed an algorithm and associated analytical methods to generate stimuli that not only minimized the impact of non-numerical magnitudes in numerosity perception but also allowed their quantification. We trained number-naïve rats with these stimuli as sound pulses representing two or three numbers and demonstrated that their numerical discrimination ability mainly relied on numerosity. Also, studying the learning process revealed that rats used numerosity before using magnitudes for choices. This numerical processing could be impaired specifically by silencing the posterior parietal cortex. Furthermore, modeling this capacity by neural networks shed light on the separation of numerosity and magnitudes extraction. Our study helps dissect the relationship between magnitude and numerosity processing, and the above different findings together affirm the independent existence of innate number and magnitudes sense in rats.

Emergence of number sense through the integration of multimodal information: developmental learning insights from neural network models

Introduction

Associating multimodal information is essential for human cognitive abilities including mathematical skills. Multimodal learning has also attracted attention in the field of machine learning, and it has been suggested that the acquisition of better latent representation plays an important role in enhancing task performance. This study aimed to explore the impact of multimodal learning on representation, and to understand the relationship between multimodal representation and the development of mathematical skills.

Methods

We employed a multimodal deep neural network as the computational model for multimodal associations in the brain. We compared the representations of numerical information, that is, handwritten digits and images containing a variable number of geometric figures learned through single- and multimodal methods. Next, we evaluated whether these representations were beneficial for downstream arithmetic tasks.

Results

Multimodal training produced better latent representation in terms of clustering quality, which is consistent with previous findings on multimodal learning in deep neural networks. Moreover, the representations learned using multimodal information exhibited superior performance in arithmetic tasks.

Discussion

Our novel findings experimentally demonstrate that changes in acquired latent representations through multimodal association learning are directly related to cognitive functions, including mathematical skills. This supports the possibility that multimodal learning using deep neural network models may offer novel insights into higher cognitive functions.

Spontaneous emergence of rudimentary music detectors in deep neural networks

Music exists in almost every society, has universal acoustic features, and is processed by distinct neural circuits in humans even with no experience of musical training. However, it remains unclear how these innate characteristics emerge and what functions they serve. Here, using an artificial deep neural network that models the auditory information processing of the brain, we show that units tuned to music can spontaneously emerge by learning natural sound detection, even without learning music. The music-selective units encoded the temporal structure of music in multiple timescales, following the population-level response characteristics observed in the brain. We found that the process of generalization is critical for the emergence of music-selectivity and that music-selectivity can work as a functional basis for the generalization of natural sound, thereby elucidating its origin. These findings suggest that evolutionary adaptation to process natural sounds can provide an initial blueprint for our sense of music.

Effects of spatial frequency cross-adaptation on the visual number sense

When observing a simple visual scene such as an array of dots, observers can easily and automatically extract their number. How does our visual system accomplish this? We investigate the role of specific spatial frequencies to the encoding of number through cross-adaptation. In two experiments, observers were peripherally adapted to six randomly generated sinusoidal gratings varying from relatively low-spatial frequency (M = 0.44 c/deg) to relatively high-spatial frequency (M = 5.88 c/deg). Subsequently, observers judged which side of the screen had a higher number of dots. We found a strong number-adaptation effect to low-spatial frequency gratings (i.e., participants significantly underestimated the number of dots on the adapted side) but a significantly reduced adaptation effect for high-spatial frequency gratings. Various control conditions demonstrate that these effects are not due to a generic response bias for the adapted side, nor moderated by dot size or spacing effects. In a third experiment, we observed no cross-adaptation for centrally presented gratings. Our results show that observers' peripheral number perception can be adapted even with stimuli lacking any numeric or segmented object information and that low spatial frequencies adapt peripheral number perception more than high ones. Together, our results are consistent with recent number perception models that suggest a key role for spatial frequency in the extraction of number from the visual signal (e.g., Paul, Ackooij, Ten Cate, & Harvey, 2022), but additionally suggest that some spatial frequencies - especially in the low range and in the periphery - may be weighted more by the visual system when estimating number. We argue that the cross-adaptation paradigm is also a useful methodology for discovering the primitives of visual number encoding.

Efficient, continual, and generalized learning in the brain - neural mechanism of Mental Schema 2.0

There has been tremendous progress in artificial neural networks (ANNs) over the past decade; however, the gap between ANNs and the biological brain as a learning device remains large. With the goal of closing this gap, this paper reviews learning mechanisms in the brain by focusing on three important issues in ANN research: efficiency, continuity, and generalization. We first discuss the method by which the brain utilizes a variety of self-organizing mechanisms to maximize learning efficiency, with a focus on the role of spontaneous activity of the brain in shaping synaptic connections to facilitate spatiotemporal learning and numerical processing. Then, we examined the neuronal mechanisms that enable lifelong continual learning, with a focus on memory replay during sleep and its implementation in brain-inspired ANNs. Finally, we explored the method by which the brain generalizes learned knowledge in new situations, particularly from the mathematical generalization perspective of topology. Besides a systematic comparison in learning mechanisms between the brain and ANNs, we propose "Mental Schema 2.0," a new computational property underlying the brain's unique learning ability that can be implemented in ANNs.

Numerical Representation for Action in Crows Obeys the Weber-Fechner Law

The psychophysical laws governing the judgment of perceived numbers of objects or events, called the number sense, have been studied in detail. However, the behavioral principles of equally important numerical representations for action are largely unexplored in both humans and animals. We trained two male carrion crows (Corvus corone) to judge numerical values of instruction stimuli from one to five and to flexibly perform a matching number of pecks. Our quantitative analysis of the crows' number production performance shows the same behavioral regularities that have previously been demonstrated for the judgment of sensory numerosity, such as the numerical distance effect, the numerical magnitude effect, and the logarithmical compression of the number line. The presence of these psychophysical phenomena in crows producing number of pecks suggests a unified sensorimotor number representation system underlying the judgment of the number of external stimuli and internally generated actions.

Emergence of behavioral phenomena and adaptation effects in human numerosity decoder using recurrent neural networks

Humans possess an innate ability to visually perceive numerosities, which refers to the cardinality of a set. Numerous studies indicate that the lateral intraparietal cortex (LIP) and other intraparietal sulcus (IPS) regions (region) of the brain contain the neurological substrates responsible for number processing. Existing computational models of number perception often focus on a limited range of numbers and fail to account for important behavioral characteristics like adaptation effects, despite simulating fundamental aspects such as size and distance effects. To address these limitations, our study develops (introduces) a novel computational model of number perception utilizing a network of neurons with self-excitatory and mutual inhibitory properties. Our approach assumes that the mean activation of the network at steady state can encode numerosity by exhibiting a monotonically increasing relationship with the input variable set size. By optimizing the total number of inhibition strengths required, we achieve coverage of the full range of numbers through three distinct intervals: 1 to 4, 5 to 17, and 21 to 50. Remarkably, this division aligns closely with the breakpoints in numerosity perception identified in behavioral studies. Furthermore, our study develops a method for decoding the mean activation into a continuous scale of numbers spanning from 1 to 50. Additionally, we propose a mechanism for dynamically selecting the inhibition strength based on current inputs, enabling the network to operate effectively across an extended (entire) range of numerosities. Our model not only sheds new light on the generation of diverse behavioral phenomena in the brain but also elucidates how continuous visual attributes and adaptation effects influence perceived numerosity.

Intelligent Recognition Using Ultralight Multifunctional Nano-Layered Carbon Aerogel Sensors with Human-Like Tactile Perception

Humans can perceive our complex world through multi-sensory fusion. Under limited visual conditions, people can sense a variety of tactile signals to identify objects accurately and rapidly. However, replicating this unique capability in robots remains a significant challenge. Here, we present a new form of ultralight multifunctional tactile nano-layered carbon aerogel sensor that provides pressure, temperature, material recognition and 3D location capabilities, which is combined with multimodal supervised learning algorithms for object recognition. The sensor exhibits human-like pressure (0.04-100 kPa) and temperature (21.5-66.2 °C) detection, millisecond response times (11 ms), a pressure sensitivity of 92.22 kPa-1 and triboelectric durability of over 6000 cycles. The devised algorithm has universality and can accommodate a range of application scenarios. The tactile system can identify common foods in a kitchen scene with 94.63% accuracy and explore the topographic and geomorphic features of a Mars scene with 100% accuracy. This sensing approach empowers robots with versatile tactile perception to advance future society toward heightened sensing, recognition and intelligence.

Are humans still necessary?

Our long accepted and historically-persistent human narrative almost exclusively places us at the motivational centre of events. The wellspring of this anthropocentric fable arises from the unitary and bounded nature of personal consciousness. Such immediate conscious experience frames the heroic vision we have told to, and subsequently sold to ourselves. But need this centrality necessarily be a given? The following work challenges this, oft unquestioned, foundational assumption, especially in light of developments in automated, autonomous, and artificially-intelligent systems. For, in these latter technologies, human contributions are becoming ever more peripheral and arguably unnecessary. The removal of the human operator from the inner loops of momentary control has progressed to now an ever more remote function as some form of supervisory monitor. The natural progression of that line of evolution is the eventual excision of humans from access to any form of control loop at all. This may even include system maintenance and then, prospectively, even initial design. The present argument features a 'unit of analysis' provocation which explores the proposition that socially, and even ergonomically, the human individual no longer occupies priority or any degree of pre-eminent centrality. Rather, we are witnessing a transitional phase of development in which socio-technical collectives are evolving as the principle sources of what, may well be profoundly unhuman motivation. These developing proclivities occupy our landscape of technological innovations that daily act to magnify, rather than diminish, such progressive inhumanities. Where this leaves a science focused on work as a human-centred enterprise serves to occupy the culminating consideration of the present discourse.

Neural coding of numerousness

Perception of numerousness, i.e. number of items in a set, is an important cognitive ability, which is present in several animal taxa. In spite of obvious differences in neuroanatomy, insects, fishes, reptiles, birds, and mammals all possess a "number sense". Furthermore, information regarding numbers can belong to different sensory modalities: animals can estimate a number of visual items, a number of tones, or a number of their own movements. Given both the heterogeneity of stimuli and of the brains processing these stimuli, it is hard to imagine that number cognition can be traced back to the same evolutionary conserved neural pathway. However, neurons that selectively respond to the number of stimuli have been described in higher-order integration brain centres both in primates and in birds, two evolutionary distant groups. Although most probably not of the same evolutionary origin, these number neurons share remarkable similarities in their response properties. Instead of homology, this similarity might result from computational advantages of the underlying coding mechanism. This means that one might expect numerousness information to undergo similar steps of neural processing even in evolutionary distant neural pathways. Following this logic, in this review we summarize our current knowledge of how numerousness is processed in the brain from sensory input to coding of abstract information in the higher-order integration centres. We also propose a list of key open questions that might promote future research on number cognition.

Fundamental units of numerosity estimation

Humans can approximately enumerate a large number of objects at a single glance. While several mechanisms have been proposed to account for this ability, the fundamental units over which they operate remain unclear. Previous studies have argued that estimation mechanisms act only on topologically distinct units or on units formed by spatial grouping cues such as proximity and connectivity, but not on units grouped by similarity. Over four experiments, we tested this claim by systematically assessing and demonstrating that similarity grouping leads to underestimation, just as spatial grouping does. Ungrouped objects with the same low-level properties as grouped objects did not cause underestimation. Further, the underestimation caused by spatial and similarity grouping was additive, suggesting that these grouping processes operate independently. These findings argue against the proposal that estimation mechanisms operate solely on topological units. Instead, we conclude that estimation processes act on representations constructed after Gestalt grouping principles, whether similarity based or spatial, have organised incoming visual input.

Comparison of visual quantities in untrained neural networks

The ability to compare quantities of visual objects with two distinct measures, proportion and difference, is observed even in newborn animals. However, how this function originates in the brain, even before visual experience, remains unknown. Here, we propose a model in which neuronal tuning for quantity comparisons can arise spontaneously in completely untrained neural circuits. Using a biologically inspired model neural network, we find that single units selective to proportions and differences between visual quantities emerge in randomly initialized feedforward wirings and that they enable the network to perform quantity comparison tasks. Notably, we find that two distinct tunings to proportion and difference originate from a random summation of monotonic, nonlinear neural activities and that a slight difference in the nonlinear response function determines the type of measure. Our results suggest that visual quantity comparisons are primitive types of functions that can emerge spontaneously before learning in young brains.

Visual number sense for real-world scenes shared by deep neural networks and humans

Recently, visual number sense has been identified from deep neural networks (DNNs). However, whether DNNs have the same capacity for real-world scenes, rather than the simple geometric figures that are often tested, is unclear. In this study, we explore the number perception of scenes using AlexNet and find that numerosity can be represented by the pattern of group activation of the category layer units. The global activation of these units increases with the number of objects in the scene, and the variations in their activation decrease accordingly. By decoding the numerosity from this pattern, we reveal that the embedding coefficient of a scene determines the likelihood of potential objects to contribute to numerical perception. This was demonstrated by the more optimized performance for pictures with relatively high embedding coefficients in both DNNs and humans. This study for the first time shows that a distinct feature in visual environments, revealed by DNNs, can modulate human perception, supported by a group-coding mechanism.

Learning-induced reorganization of number neurons and emergence of numerical representations in a biologically inspired neural network

Number sense, the ability to decipher quantity, forms the foundation for mathematical cognition. How number sense emerges with learning is, however, not known. Here we use a biologically-inspired neural architecture comprising cortical layers V1, V2, V3, and intraparietal sulcus (IPS) to investigate how neural representations change with numerosity training. Learning dramatically reorganized neuronal tuning properties at both the single unit and population levels, resulting in the emergence of sharply-tuned representations of numerosity in the IPS layer. Ablation analysis revealed that spontaneous number neurons observed prior to learning were not critical to formation of number representations post-learning. Crucially, multidimensional scaling of population responses revealed the emergence of absolute and relative magnitude representations of quantity, including mid-point anchoring. These learnt representations may underlie changes from logarithmic to cyclic and linear mental number lines that are characteristic of number sense development in humans. Our findings elucidate mechanisms by which learning builds novel representations supporting number sense.

Categorical representation of abstract spatial magnitudes in the executive telencephalon of crows

The ability to group abstract continuous magnitudes into meaningful categories is cognitively demanding but key to intelligent behavior. To explore its neuronal mechanisms, we trained carrion crows to categorize lines of variable lengths into arbitrary "short" and "long" categories. Single-neuron activity in the nidopallium caudolaterale (NCL) of behaving crows reflected the learned length categories of visual stimuli. The length categories could be reliably decoded from neuronal population activity to predict the crows' conceptual decisions. NCL activity changed with learning when a crow was retrained with the same stimuli assigned to more categories with new boundaries ("short", "medium," and "long"). Categorical neuronal representations emerged dynamically so that sensory length information at the beginning of the trial was transformed into behaviorally relevant categorical representations shortly before the crows' decision making. Our data show malleable categorization capabilities for abstract spatial magnitudes mediated by the flexible networks of the crow NCL.

Artificial neural network modelling of the neural population code underlying mathematical operations

Mathematical operations have long been regarded as a sparse, symbolic process in neuroimaging studies. In contrast, advances in artificial neural networks (ANN) have enabled extracting distributed representations of mathematical operations. Recent neuroimaging studies have compared distributed representations of the visual, auditory and language domains in ANNs and biological neural networks (BNNs). However, such a relationship has not yet been examined in mathematics. Here we hypothesise that ANN-based distributed representations can explain brain activity patterns of symbolic mathematical operations. We used the fMRI data of a series of mathematical problems with nine different combinations of operators to construct voxel-wise encoding/decoding models using both sparse operator and latent ANN features. Representational similarity analysis demonstrated shared representations between ANN and BNN, an effect particularly evident in the intraparietal sulcus. Feature-brain similarity (FBS) analysis served to reconstruct a sparse representation of mathematical operations based on distributed ANN features in each cortical voxel. Such reconstruction was more efficient when using features from deeper ANN layers. Moreover, latent ANN features allowed the decoding of novel operators not used during model training from brain activity. The current study provides novel insights into the neural code underlying mathematical thought.

Using artificial neural networks to ask 'why' questions of minds and brains

Neuroscientists have long characterized the properties and functions of the nervous system, and are increasingly succeeding in answering how brains perform the tasks they do. But the question 'why' brains work the way they do is asked less often. The new ability to optimize artificial neural networks (ANNs) for performance on human-like tasks now enables us to approach these 'why' questions by asking when the properties of networks optimized for a given task mirror the behavioral and neural characteristics of humans performing the same task. Here we highlight the recent success of this strategy in explaining why the visual and auditory systems work the way they do, at both behavioral and neural levels.

Quantum spin models for numerosity perception

Humans share with animals, both vertebrates and invertebrates, the capacity to sense the number of items in their environment already at birth. The pervasiveness of this skill across the animal kingdom suggests that it should emerge in very simple populations of neurons. Current modelling literature, however, has struggled to provide a simple architecture carrying out this task, with most proposals suggesting the emergence of number sense in multi-layered complex neural networks, and typically requiring supervised learning; while simple accumulator models fail to predict Weber's Law, a common trait of human and animal numerosity processing. We present a simple quantum spin model with all-to-all connectivity, where numerosity is encoded in the spectrum after stimulation with a number of transient signals occurring in a random or orderly temporal sequence. We use a paradigmatic simulational approach borrowed from the theory and methods of open quantum systems out of equilibrium, as a possible way to describe information processing in neural systems. Our method is able to capture many of the perceptual characteristics of numerosity in such systems. The frequency components of the magnetization spectra at harmonics of the system's tunneling frequency increase with the number of stimuli presented. The amplitude decoding of each spectrum, performed with an ideal-observer model, reveals that the system follows Weber's law. This contrasts with the well-known failure to reproduce Weber's law with linear system or accumulators models.

A neural theory for counting memories

Keeping track of the number of times different stimuli have been experienced is a critical computation for behavior. Here, we propose a theoretical two-layer neural circuit that stores counts of stimulus occurrence frequencies. This circuit implements a data structure, called a count sketch, that is commonly used in computer science to maintain item frequencies in streaming data. Our first model implements a count sketch using Hebbian synapses and outputs stimulus-specific frequencies. Our second model uses anti-Hebbian plasticity and only tracks frequencies within four count categories ("1-2-3-many"), which trades-off the number of categories that need to be distinguished with the potential ethological value of those categories. We show how both models can robustly track stimulus occurrence frequencies, thus expanding the traditional novelty-familiarity memory axis from binary to discrete with more than two possible values. Finally, we show that an implementation of the "1-2-3-many" count sketch exists in the insect mushroom body.

Assessing the ecological validity of numerosity-selective neuronal populations with real-world natural scenes

Animals and humans are able to quickly and effortlessly estimate the number of items in a set: their numerosity. Numerosity perception is thought to be critical to behavior, from feeding to escaping predators to human mathematical cognition. Virtually, all scientific studies on numerosity mechanisms use well controlled but artificial stimuli to isolate the numerosity dimension from other physical quantities. Here, we probed the ecological validity of these artificial stimuli and evaluate whether an important component in numerosity processing, the numerosity-selective neural populations, also respond to numerosity of items in real-world natural scenes. Using 7T MRI and natural images from a wide range of categories, we provide evidence that the numerosity-tuned neuronal populations show numerosity-selective responses when viewing images from a real-world natural scene. Our findings strengthen the role of numerosity-selective neurons in numerosity perception and provide an important link to their function in numerosity perception in real-world settings.

General object-based features account for letter perception

After years of experience, humans become experts at perceiving letters. Is this visual capacity attained by learning specialized letter features, or by reusing general visual features previously learned in service of object categorization? To explore this question, we first measured the perceptual similarity of letters in two behavioral tasks, visual search and letter categorization. Then, we trained deep convolutional neural networks on either 26-way letter categorization or 1000-way object categorization, as a way to operationalize possible specialized letter features and general object-based features, respectively. We found that the general object-based features more robustly correlated with the perceptual similarity of letters. We then operationalized additional forms of experience-dependent letter specialization by altering object-trained networks with varied forms of letter training; however, none of these forms of letter specialization improved the match to human behavior. Thus, our findings reveal that it is not necessary to appeal to specialized letter representations to account for perceptual similarity of letters. Instead, we argue that it is more likely that the perception of letters depends on domain-general visual features.

For over a century, scientists have conducted behavioral experiments to investigate how the visual system recognizes letters, but it has proven difficult to propose a model of the feature space underlying this capacity. Here we leveraged recent advances in machine learning to model a wide variety of features ranging from specialized letter features to general object-based features. Across two large-scale behavioral experiments we find that general object-based features account well for letter perception, and that adding letter specialization did not improve the correspondence to human behavior. It is plausible that the ability to recognize letters largely relies on general visual features unaltered by letter learning.

Recognition ability of untrained neural networks to symbolic numbers

Although animals can learn to use abstract numbers to represent the number of items, whether untrained animals could distinguish between different abstract numbers is not clear. A two-layer spiking neural network with lateral inhibition was built from the perspective of biological interpretability. The network connection weight was set randomly without adjustment. On the basis of this model, experiments were carried out on the symbolic number dataset MNIST and non-symbolic numerosity dataset. Results showed that the model has abilities to distinguish symbolic numbers. However, compared with number sense, tuning curves of symbolic numbers could not reproduce size and distance effects. The preference distribution also could not show high distribution characteristics at both ends and low distribution characteristics in the middle. More than half of the network units prefer the symbolic numbers 0 and 5. The average goodness-of-fit of the Gaussian fitting of tuning curves increases with the increase in abscissa non-linearity. These results revealed that the concept of human symbolic number is trained on the basis of number sense.

The malleable impact of non-numeric features in visual number perception

Non-numeric stimulus features frequently influence observers' number judgments: when judging the number of items in a display, we will often (mis)perceive the set with a larger cumulative surface area as more numerous. These "congruency effects" are often used as evidence for how vision extracts numeric information and have been invoked in arguments surrounding whether non-numeric cues (e.g., cumulative area, density, etc.) are combined for number perception. We test whether congruency effects for one such cue - cumulative area - provide evidence that it is necessarily used and integrated in number perception, or if its influence on number is malleable. In Experiment 1, we replicate and extend prior work showing that the presence of feedback eliminates congruency effects between number and cumulative area, suggesting that the role of cumulative area in number perception is malleable rather than obligatory. In Experiment 2, we test whether this malleable influence is because of use of prior experiences about how number naturalistically correlates with cumulative area, or the result of response competition, with number and cumulative area actively competing for the same behavioral decision. We preserve cumulative area as a visual cue but eliminate response competition with number by replacing one side of the dot array with its corresponding Hindu-Arabic numeral. Independent of the presence or absence of feedback, we do not observe congruency effects in Experiment 2. These experiments suggest that cumulative area is not necessarily integrated in number perception nor a reflection of a rational use of naturalistic correlations, but rather congruency effects between cumulative area and number emerge as a consequence of response competition. Our findings help to elucidate the mechanism through which non-numeric cues and number interact, and provide an explanation for why congruency effects are only sometimes observed across studies.

Number neurons in the nidopallium of young domestic chicks

Numerosity, that is, the number of items in a set, is a significant aspect in the perception of the environment. Behavioral and in silico experiments suggest that number sense belongs to a core knowledge system and can be present already at birth. However, neurons sensitive to the number of visual items have been so far described only in the brain of adult animals. Therefore, it remained unknown to what extent their selectivity would depend on visual learning and experience. We found number neurons in the caudal nidopallium (a higher associative area functionally similar to the mammalian prefrontal cortex) of very young, numerically naïve domestic chicks. This result suggests that numerosity perception is possibly an inborn feature of the vertebrate brain.

Numerical cognition is ubiquitous in the animal kingdom. Domestic chicks are a widely used developmental model for studying numerical cognition. Soon after hatching, chicks can perform sophisticated numerical tasks. Nevertheless, the neural basis of their numerical abilities has remained unknown. Here, we describe number neurons in the caudal nidopallium (functionally equivalent to the mammalian prefrontal cortex) of young domestic chicks. Number neurons that we found in young chicks showed remarkable similarities to those in the prefrontal cortex and caudal nidopallium of adult animals. Thus, our results suggest that numerosity perception based on number neurons might be an inborn feature of the vertebrate brain.

Attention drives human numerosity-selective responses

Numerosity, the set size of a group of items, helps guide behavior and decisions. Previous studies have shown that neural populations respond selectively to numerosities. How numerosity is extracted from the visual scene is a longstanding debate, often contrasting low-level visual with high-level cognitive processes. Here, we investigate how attention influences numerosity-selective responses. The stimuli consisted of black and white dots within the same display. Participants' attention was focused on either black or white dots, while we systematically changed the numerosity of black, white, and total dots. Using 7 T fMRI, we show that the numerosity-tuned neural populations respond only when attention is focused on their preferred numerosity, irrespective of the unattended or total numerosities. Without attention, responses to preferred numerosity are suppressed. Unlike traditional effects of attention in the visual cortex, where attention enhances already existing responses, these results suggest that attention is required to drive numerosity-selective responses.

Role of Lateral Inhibition on Visual Number Sense

Newborn animals, such as 4-month-old infants, 4-day-old chicks, and 1-day-old guppies, exhibit sensitivity to an approximate number of items in the visual array. These findings are often interpreted as evidence for an innate "number sense." However, number sense is typically investigated using explicit behavioral tasks, which require a form of calibration (e.g., habituation or reward-based training) in experimental studies. Therefore, the generation of number sense may be the result of calibration. We built a number-sense neural network model on the basis of lateral inhibition to explore whether animals demonstrate an innate "number sense" and determine important factors affecting this competence. The proposed model can reproduce size and distance effects of output responses of number-selective neurons when network connection weights are set randomly without an adjustment. Results showed that number sense can be produced under the influence of lateral inhibition, which is one of the fundamental mechanisms of the nervous system, and independent of learning.

Emerged human-like facial expression representation in a deep convolutional neural network

Recent studies found that the deep convolutional neural networks (DCNNs) trained to recognize facial identities spontaneously learned features that support facial expression recognition, and vice versa. Here, we showed that the self-emerged expression-selective units in a VGG-Face trained for facial identification were tuned to distinct basic expressions and, importantly, exhibited hallmarks of human expression recognition (i.e., facial expression confusion and categorical perception). We then investigated whether the emergence of expression-selective units is attributed to either face-specific experience or domain-general processing by conducting the same analysis on a VGG-16 trained for object classification and an untrained VGG-Face without any visual experience, both having the identical architecture with the pretrained VGG-Face. Although similar expression-selective units were found in both DCNNs, they did not exhibit reliable human-like characteristics of facial expression perception. Together, these findings revealed the necessity of domain-specific visual experience of face identity for the development of facial expression perception, highlighting the contribution of nurture to form human-like facial expression perception.

Brain-like functional specialization emerges spontaneously in deep neural networks

The human brain contains multiple regions with distinct, often highly specialized functions, from recognizing faces to understanding language to thinking about what others are thinking. However, it remains unclear why the cortex exhibits this high degree of functional specialization in the first place. Here, we consider the case of face perception using artificial neural networks to test the hypothesis that functional segregation of face recognition in the brain reflects a computational optimization for the broader problem of visual recognition of faces and other visual categories. We find that networks trained on object recognition perform poorly on face recognition and vice versa and that networks optimized for both tasks spontaneously segregate themselves into separate systems for faces and objects. We then show functional segregation to varying degrees for other visual categories, revealing a widespread tendency for optimization (without built-in task-specific inductive biases) to lead to functional specialization in machines and, we conjecture, also brains.

Numerosity tuning in human association cortices and local image contrast representations in early visual cortex

Human early visual cortex response amplitudes monotonically increase with numerosity (object number), regardless of object size and spacing. However, numerosity is typically considered a high-level visual or cognitive feature, while early visual responses follow image contrast in the spatial frequency domain. We find that, at fixed contrast, aggregate Fourier power (at all orientations and spatial frequencies) follows numerosity closely but nonlinearly with little effect of object size, spacing or shape. This would allow straightforward numerosity estimation from spatial frequency domain image representations. Using 7T fMRI, we show monotonic responses originate in primary visual cortex (V1) at the stimulus's retinotopic location. Responses here and in neural network models follow aggregate Fourier power more closely than numerosity. Truly numerosity tuned responses emerge after lateral occipital cortex and are independent of retinotopic location. We propose numerosity's straightforward perception and neural responses may result from the pervasive spatial frequency analyses of early visual processing.

A connectome-based neuromarker of nonverbal number acuity and arithmetic skills

The approximate number system (ANS) is vital for survival and reproduction in animals and is crucial for constructing abstract mathematical abilities in humans. Most previous neuroimaging studies focused on identifying discrete brain regions responsible for the ANS and characterizing their functions in numerosity perception. However, a neuromarker to characterize an individual's ANS acuity is lacking, especially one based on whole-brain functional connectivity (FC). Here, based on the resting-state functional magnetic resonance imaging (rs-fMRI) data obtained from a large sample, we identified a distributed brain network (i.e. a numerosity network) using a connectome-based predictive modeling (CPM) analysis. The summed FC strength within the numerosity network reliably predicted individual differences in ANS acuity regarding behavior, as measured using a nonsymbolic number-comparison task. Furthermore, in an independent dataset of the Human Connectome Project (HCP), we found that the summed FC strength within the numerosity network also specifically predicted individual differences in arithmetic skills, but not domain-general cognitive abilities. Therefore, our findings revealed that the identified numerosity network could serve as an applicable neuroimaging-based biomarker of nonverbal number acuity and arithmetic skills.

Electrophysiological Signatures of Numerosity Encoding in a Delayed Match-to-Sample Task

The number of elements in a small set of items is appraised in a fast and exact manner, a phenomenon called subitizing. In contrast, humans provide imprecise responses when comparing larger numerosities, with decreasing precision as the number of elements increases. Estimation is thought to rely on a dedicated system for the approximate representation of numerosity. While previous behavioral and neuroimaging studies associate subitizing to a domain-general system related to object tracking and identification, the nature of small numerosity processing is still debated. We investigated the neural processing of numerosity across subitizing and estimation ranges by examining electrophysiological activity during the memory retention period in a delayed numerical match-to-sample task. We also assessed potential differences in the neural signature of numerical magnitude in a fully non-symbolic or cross-format comparison. In line with behavioral performance, we observed modulation of parietal-occipital neural activity as a function of numerosity that differed in two ranges, with distinctive neural signatures of small numerosities showing clear similarities with those observed in visuospatial working memory tasks. We also found differences in neural activity related to numerical information in anticipation of single vs. cross-format comparison, suggesting a top-down modulation of numerical processing. Finally, behavioral results revealed enhanced performance in the mixed-format conditions and a significant correlation between task performance and symbolic mathematical skills. Overall, we provide evidence for distinct mechanisms related to small and large numerosity and differences in numerical encoding based on task demands.

Visual Search Asymmetry: Deep Nets and Humans Share Similar Inherent Biases

Visual search is a ubiquitous and often challenging daily task, exemplified by looking for the car keys at home or a friend in a crowd. An intriguing property of some classical search tasks is an asymmetry such that finding a target A among distractors B can be easier than finding B among A. To elucidate the mechanisms responsible for asymmetry in visual search, we propose a computational model that takes a target and a search image as inputs and produces a sequence of eye movements until the target is found. The model integrates eccentricity-dependent visual recognition with target-dependent top-down cues. We compared the model against human behavior in six paradigmatic search tasks that show asymmetry in humans. Without prior exposure to the stimuli or task-specific training, the model provides a plausible mechanism for search asymmetry. We hypothesized that the polarity of search asymmetry arises from experience with the natural environment. We tested this hypothesis by training the model on augmented versions of ImageNet where the biases of natural images were either removed or reversed. The polarity of search asymmetry disappeared or was altered depending on the training protocol. This study highlights how classical perceptual properties can emerge in neural network models, without the need for task-specific training, but rather as a consequence of the statistical properties of the developmental diet fed to the model. All source code and data are publicly available at https://github.com/kreimanlab/VisualSearchAsymmetry.

Spontaneous representation of numerosity zero in a deep neural network for visual object recognition

Conceiving "nothing" as a numerical value zero is considered a sophisticated numerical capability that humans share with cognitively advanced animals. We demonstrate that representation of zero spontaneously emerges in a deep learning neural network without any number training. As a signature of numerical quantity representation, and similar to real neurons from animals, numerosity zero network units show maximum activity to empty sets and a gradual decrease in activity with increasing countable numerosities. This indicates that the network spontaneously ordered numerosity zero as the smallest numerical value along the number line. Removal of empty-set network units caused specific deficits in the network's judgment of numerosity zero, thus reflecting these units' functional relevance. These findings suggest that processing visual information is sufficient for a visual number sense that includes zero to emerge and explains why cognitively advanced animals with whom we share a nonverbal number system exhibit rudiments of numerosity zero.

The role of neural tuning in quantity perception

Perception of quantities, such as numerosity, timing, and size, is essential for behavior and cognition. Accumulating evidence demonstrates neurons processing quantities are tuned, that is, have a preferred quantity amount, not only for numerosity, but also other quantity dimensions and sensory modalities. We argue that quantity-tuned neurons are fundamental to understanding quantity perception. We illustrate how the properties of quantity-tuned neurons can underlie a range of perceptual phenomena. Furthermore, quantity-tuned neurons are organized in distinct but overlapping topographic maps. We suggest that this overlap in tuning provides the neural basis for perceptual interactions between different quantities, without the need for a common neural representational code.

Learning exact enumeration and approximate estimation in deep neural network models

A system for approximate number discrimination has been shown to arise in at least two types of hierarchical neural network models-a generative Deep Belief Network (DBN) and a Hierarchical Convolutional Neural Network (HCNN) trained to classify natural objects. Here, we investigate whether the same two network architectures can learn to recognise exact numerosity. A clear difference in performance could be traced to the specificity of the unit responses that emerged in the last hidden layer of each network. In the DBN, the emergence of a layer of monotonic 'summation units' was sufficient to produce classification behaviour consistent with the behavioural signature of the approximate number system. In the HCNN, a layer of units uniquely tuned to the transition between particular numerosities effectively encoded a thermometer-like 'numerosity code' that ensured near-perfect classification accuracy. The results support the notion that parallel pattern-recognition mechanisms may give rise to exact and approximate number concepts, both of which may contribute to the learning of symbolic numbers and arithmetic.

Numerosity representation in a deep convolutional neural network

Enumerating objects in the environment (i.e., “number sense”) is crucial for survival in many animal species, and foundational for the construction of more abstract and complex mathematical knowledge in humans. Perhaps surprisingly, deep convolutional neural networks (DCNNs) spontaneously emerge a similar number sense even without any explicit training for numerosity estimation. However, little is known about how the number sense emerges, and the extent to which it is comparable with human number sense. Here, we examined whether the numerosity underestimation effect, a phenomenon indicating that numerosity perception acts upon the perceptual number rather than the physical number, can be observed in DCNNs. In a typical DCNN, AlexNet, we found that number-selective units at late layers operated on the perceptual number, like humans do. More importantly, this perceptual number sense did not emerge abruptly, rather developed progressively along the hierarchy in the DCNN, shifting from the physical number sense at early layers to perceptual number sense at late layers. Our finding hence provides important implications for the neural implementation of number sense in the human brain and advocates future research to determine whether the representation of numerosity also develops gradually along the human visual stream from physical number to perceptual number.