Visual prototypes in the ventral stream are attuned to complexity and gaze behavior

Emergence of Emotion Selectivity in Deep Neural Networks Trained to Recognize Visual Objects

Recent neuroimaging studies have shown that the visual cortex plays an important role in representing the affective significance of visual input. The origin of these affect-specific visual representations is debated: they are intrinsic to the visual system versus they arise through reentry from frontal emotion processing structures such as the amygdala. We examined this problem by combining convolutional neural network (CNN) models of the human ventral visual cortex pre-trained on ImageNet with two datasets of affective images. Our results show that in all layers of the CNN models, there were artificial neurons that responded consistently and selectively to neutral, pleasant, or unpleasant images and lesioning these neurons by setting their output to zero or enhancing these neurons by increasing their gain led to decreased or increased emotion recognition performance respectively. These results support the idea that the visual system may have the intrinsic ability to represent the affective significance of visual input and suggest that CNNs offer a fruitful platform for testing neuroscientific theories.

Representation of Natural Contours by a Neural Population in Monkey V4

The cortical visual area, V4, has been considered to code contours that contribute to the intermediate-level representation of objects. The neural responses to the complex contour features intrinsic to natural contours are expected to clarify the essence of the representation. To approach the cortical coding of natural contours, we investigated the simultaneous coding of multiple contour features in monkey (Macaca fuscata) V4 neurons and their population-level representation. A substantial number of neurons showed significant tuning for two or more features such as curvature and closure, indicating that a substantial number of V4 neurons simultaneously code multiple contour features. A large portion of the neurons responded vigorously to acutely curved contours that surrounded the center of classical receptive field, suggesting that V4 neurons tend to code prominent features of object contours. The analysis of mutual information (MI) between the neural responses and each contour feature showed that most neurons exhibited similar magnitudes for each type of MI, indicating that many neurons showing the responses depended on multiple contour features. We next examined the population-level representation by using multidimensional scaling analysis. The neural preferences to the multiple contour features and that to natural stimuli compared with silhouette stimuli increased along with the primary and secondary axes, respectively, indicating the contribution of the multiple contour features and surface textures in the population responses. Our analyses suggested that V4 neurons simultaneously code multiple contour features in natural images and represent contour and surface properties in population.

Embracing deepfakes and AI-generated images in neuroscience research

The rise of deepfakes and AI-generated images has raised concerns regarding their potential misuse. However, this commentary highlights the valuable opportunities these technologies offer for neuroscience research. Deepfakes deliver accessible, realistic and customisable dynamic face stimuli, while generative adversarial networks (GANs) can generate and modify diverse and high-quality static content. These advancements can enhance the variability and ecological validity of research methods and enable the creation of previously unattainable stimuli. When AI-generated images are informed by brain responses, they provide unique insights into the structure and function of visual systems. The authors argue that experimental psychologists and cognitive neuroscientists stay informed about these emerging tools and embrace their potential to advance the field of visual neuroscience.

Macaques recognize features in synthetic images derived from ventral stream neurons

Primates can recognize features in virtually all types of images, an ability that still requires a comprehensive computational explanation. One hypothesis is that visual cortex neurons learn patterns from scenes, objects, and textures, and use these patterns to interpolate incoming visual information. We have used machine learning algorithms to instantiate visual patterns stored by neurons-we call these highly activating images prototypes. Prototypes from inferotemporal (IT) neurons often resemble parts of real-world objects, such as monkey faces and body parts, a similarity established via pretrained neural networks [C. R. Ponce et al., Cell 177, 999-1009.e10 (2019)] and naïve human participants [A. Bardon, W. Xiao, C. R. Ponce, M. S. Livingstone, G. Kreiman, Proc. Natl. Acad. Sci. U.S.A. 119, e2118705119 (2022)]. However, it is not known whether monkeys themselves perceive similarities between neuronal prototypes and real-world objects. Here, we investigated whether monkeys reported similarities between prototypes and real-world objects using a two-alternative forced choice task. We trained the animals to saccade to synthetic images of monkeys, and subsequently tested how they classified prototypes synthesized from IT and primary visual cortex (V1). We found monkeys classified IT prototypes as conspecifics more often than they did random generator images and V1 prototypes, and their choices were partially predicted by convolutional neural networks. Further, we confirmed that monkeys could abstract general shape information from images of real-world objects. Finally, we verified these results with human participants. Our results provide further evidence that prototypes from cortical neurons represent interpretable abstractions from the visual world.