Inactivation of face-selective neurons alters eye movements when free viewing faces

During free viewing, faces attract gaze and induce specific fixation patterns corresponding to the facial features. This suggests that neurons encoding the facial features are in the causal chain that steers the eyes. However, there is no physiological evidence to support a mechanistic link between face-encoding neurons in high-level visual areas and the oculomotor system. In this study, we targeted the middle face patches of the inferior temporal (IT) cortex in two macaque monkeys using an functional magnetic resonance imaging (fMRI) localizer. We then utilized muscimol microinjection to unilaterally suppress IT neural activity inside and outside the face patches and recorded eye movements while the animals free viewing natural scenes. Inactivation of the face-selective neurons altered the pattern of eye movements on faces: The monkeys found faces in the scene but neglected the eye contralateral to the inactivation hemisphere. These findings reveal the causal contribution of the high-level visual cortex in eye movements.

Information content and temporal structure of face selective local field potentials frequency bands in IT cortex

Sensory stimulation triggers synchronized bioelectrical activity in the brain across various frequencies. This study delves into network-level activities, specifically focusing on local field potentials as a neural signature of visual category representation. Specifically, we studied the role of different local field potential frequency oscillation bands in visual stimulus category representation by presenting images of faces and objects to three monkeys while recording local field potential from inferior temporal cortex. We found category selective local field potential responses mainly for animate, but not inanimate, objects. Notably, face-selective local field potential responses were evident across all tested frequency bands, manifesting in both enhanced (above mean baseline activity) and suppressed (below mean baseline activity) local field potential powers. We observed four different local field potential response profiles based on frequency bands and face selective excitatory and suppressive responses. Low-frequency local field potential bands (1-30 Hz) were more prodominstaly suppressed by face stimulation than the high-frequency (30-170 Hz) local field potential bands. Furthermore, the low-frequency local field potentials conveyed less face category informtion than the high-frequency local field potential in both enhansive and suppressive conditions. Furthermore, we observed a negative correlation between face/object d-prime values in all the tested local field potential frequency bands and the anterior-posterior position of the recording sites. In addition, the power of low-frequency local field potential systematically declined across inferior temporal anterior-posterior positions, whereas high-frequency local field potential did not exhibit such a pattern. In general, for most of the above-mentioned findings somewhat similar results were observed for body, but not, other stimulus categories. The observed findings suggest that a balance of face selective excitation and inhibition across time and cortical space shape face category selectivity in inferior temporal cortex.

Bodies in motion: Unraveling the distinct roles of motion and shape in dynamic body responses in the temporal cortex

The temporal cortex represents social stimuli, including bodies. We examine and compare the contributions of dynamic and static features to the single-unit responses to moving monkey bodies in and between a patch in the anterior dorsal bank of the superior temporal sulcus (dorsal patch [DP]) and patches in the anterior inferotemporal cortex (ventral patch [VP]), using fMRI guidance in macaques. The response to dynamics varies within both regions, being higher in DP. The dynamic body selectivity of VP neurons correlates with static features derived from convolutional neural networks and motion. DP neurons' dynamic body selectivity is not predicted by static features but is dominated by motion. Whereas these data support the dominance of motion in the newly proposed "dynamic social perception" stream, they challenge the traditional view that distinguishes DP and VP processing in terms of motion versus static features, underscoring the role of inferotemporal neurons in representing body dynamics.

Representational geometry of incomplete faces in macaque face patches

The neural code of faces has been intensively studied in the macaque face patch system. Although the majority of previous studies used complete faces as stimuli, faces are often seen partially in daily life. Here, we investigated how face-selective cells represent two types of incomplete faces: face fragments and occluded faces, with the location of the fragment/occluder and the facial features systematically varied. Contrary to popular belief, we found that the preferred face regions identified with two stimulus types are dissociated in many face cells. This dissociation can be explained by the nonlinear integration of information from different face parts and is closely related to a curved representation of face completeness in the state space, which allows a clear discrimination between different stimulus types. Furthermore, identity-related facial features are represented in a subspace orthogonal to the nonlinear dimension of face completeness, supporting a condition-general code of facial identity.

Temporal continuity shapes visual responses of macaque face patch neurons

Macaque inferior temporal cortex neurons respond selectively to complex visual images, with recent work showing that they are also entrained reliably by the evolving content of natural movies. To what extent does temporal continuity itself shape the responses of high-level visual neurons? We addressed this question by measuring how cells in face-selective regions of the macaque visual cortex were affected by the manipulation of a movie's temporal structure. Sampling a 5-min movie at 1 s intervals, we measured neural responses to randomized, brief stimuli of different lengths, ranging from 800 ms dynamic movie snippets to 100 ms static frames. We found that the disruption of temporal continuity strongly altered neural response profiles, particularly in the early response period after stimulus onset. The results suggest that models of visual system function based on discrete and randomized visual presentations may not translate well to the brain's natural modes of operation.

Visual recognition in rhesus monkeys requires area TE but not TEO

The primate visual system is often described as a hierarchical feature-conjunction pathway, whereby each level represents an increasingly complex combination of image elements, culminating in the representation of whole coherent images in anterior inferior temporal cortex. Although many models of the ventral visual stream emphasize serial feedforward processing ((Poggio T, Mutch J, Leibo J, Rosasco L, Tacchetti A. The computationalmagic of the ventral stream: sketch of a theory (and why some deep architectures work). TechRep MIT-CSAIL-TR-2012-035. MIT CSAIL, Cambridge, MA. 2012); (Yamins DLK, DiCarlo JJ. Eight open questions in the computational modeling of higher sensory cortex. Curr Opin Neurobiol. 2016:37:114-120.)), anatomical studies show connections that bypass intermediate areas and that feedback to preceding areas ((Distler C, Boussaoud D, Desimone R, Ungerleider LG. Cortical connections of inferior temporal area TEO in macaque monkeys. J Comp Neurol. 1993:334(1):125-150.); (Kravitz DJ, Saleem KS, Baker CI, Mishkin M. A new neural framework for visuospatial processing. Nat Rev Neurosci. 2011:12(4):217-230.)). Prior studies on visual discrimination and object transforms also provide evidence against a strictly feed-forward serial transfer of information between adjacent areas ((Kikuchi R, Iwai E. The locus of the posterior subdivision of the inferotemporal visual learning area in the monkey. Brain Res. 1980:198(2):347-360.); (Weiskrantz L, Saunders RC. Impairments of visual object transforms in monkeys. Brain. 1984:107(4):1033-1072.); (Kar K, DiCarlo JJ. Fast recurrent processing via ventrolateral prefrontal cortex is needed by the primate ventral stream for robust Core visual object recognition. Neuron. 2021:109(1):164-176.e5.)). Thus, we sought to investigate whether behaviorally relevant propagation of visual information is as strictly sequential as sometimes supposed. We compared the accuracy of visual recognition after selective removal of specific subregions of inferior temporal cortex-area TEO, area TE, or both areas combined. Removal of TEO alone had no detectable effect on recognition memory, whereas removal of TE alone produced a large and significant impairment. Combined removal of both areas created no additional deficit relative to removal of TE alone. Thus, area TE is critical for rapid visual object recognition, and detailed image-level visual information can reach area TE via a route other than through TEO.

Inferior temporal cortex leads prefrontal cortex in response to a violation of a learned sequence

Primates learn statistical regularities that are embedded in visual sequences, a form of statistical learning. Single-unit recordings in macaques showed that inferior temporal (IT) neurons are sensitive to statistical regularities in visual sequences. Here, we asked whether ventrolateral prefrontal cortex (VLPFC), which is connected to IT, is also sensitive to the transition probabilities in visual sequences and whether the statistical learning signal in IT originates in VLPFC. We recorded simultaneously multiunit activity (MUA) and local field potentials (LFPs) in IT and VLPFC after monkeys were exposed to triplets of images with a fixed presentation order. In both areas, the MUA was stronger to images that violated the learned sequence (deviants) compared to the same images presented in the learned triplets. The high-gamma and beta LFP power showed an enhanced and suppressed response, respectively, to the deviants in both areas. The enhanced response was present also for the image following the deviant, suggesting a sensitivity for temporal adjacent dependencies in IT and VLPFC. The increased response to the deviant occurred later in VLPFC than in IT, suggesting that the deviant response in IT was not inherited from VLPFC. These data support predictive coding theories that propose a feedforward flow of prediction errors.

Surgical Procedure for Implantation of Opto-Array in Nonhuman Primates

Optogenetics allows precise temporal control of neuronal activity in the brain. Engineered viral vectors are routinely used to transduce neurons with light-sensitive opsins. However, reliable virus injection and light delivery in animals with large brains, such as nonhuman primates, has proven challenging. The Opto-Array is a novel yet simple device that is used to deliver light to extended regions of the cortex surface for high-throughput behavioral optogenetics in large brains. Here we present protocols for surgical delivery of virus (Basic Protocol 1) and implantation of the Opto-Array (Basic Protocol 2) in two separate surgeries in a rhesus monkey's inferior temporal cortex. As a proof of concept, we measured the behavioral performance of an animal detecting cortical optogenetic stimulations (Basic Protocol 3) with different illumination power and duration using the Opto-Array. The animal was able to detect the optogenetic stimulation for all tested illumination powers and durations. Regression analysis also showed both power and duration of illumination significantly modulate the detectability of the optogenetic stimulation. The outcome of this approach is superior to the standard practice of injecting and recording through a chamber for large areas of the cortex surface. Moreover, the chronic nature of the Opto-Array allows perturbation of neuronal activity of the same site across multiple sessions because it is highly stable; thus, data can be pooled over months. The detailed surgical method presented here makes it possible to use optogenetics to modulate neuronal activity across large regions of the cortex surface in the nonhuman primate brain. This method also lays the groundwork for future attempts to use optogenetics to restore vision in humans. © 2023 The Authors. Current Protocols published by Wiley Periodicals LLC. Basic Protocol 1: Virus injection surgery Basic Protocol 2: Opto-Array implantation surgery Basic Protocol 3: Cortical Perturbation Detection (CPD) task behavioral training.

Encoding of 3D physical dimensions by face-selective cortical neurons

Neurons throughout the primate inferior temporal (IT) cortex respond selectively to visual images of faces and other complex objects. The response magnitude of neurons to a given image often depends on the size at which the image is presented, usually on a flat display at a fixed distance. While such size sensitivity might simply reflect the angular subtense of retinal image stimulation in degrees, one unexplored possibility is that it tracks the real-world geometry of physical objects, such as their size and distance to the observer in centimeters. This distinction bears fundamentally on the nature of object representation in IT and on the scope of visual operations supported by the ventral visual pathway. To address this question, we assessed the response dependency of neurons in the macaque anterior fundus (AF) face patch to the angular versus physical size of faces. We employed a macaque avatar to stereoscopically render three-dimensional (3D) photorealistic faces at multiple sizes and distances, including a subset of size/distance combinations designed to cast the same size retinal image projection. We found that most AF neurons were modulated principally by the 3D physical size of the face rather than its two-dimensional (2D) angular size on the retina. Further, most neurons responded strongest to extremely large and small faces, rather than to those of normal size. Together, these findings reveal a graded encoding of physical size among face patch neurons, providing evidence that category-selective regions of the primate ventral visual pathway participate in a geometric analysis of real-world objects.

Image-dependence of the detectability of optogenetic stimulation in macaque inferotemporal cortex

Artificial activation of neurons in early visual areas induces perception of simple visual flashes.1,2 Accordingly, stimulation in high-level visual cortices is expected to induce perception of complex features.3,4 However, results from studies in human patients challenge this expectation. Stimulation rarely induces any detectable visual event, and never a complex one, in human subjects with closed eyes.2 Stimulation of the face-selective cortex in a human patient led to remarkable hallucinations only while the subject was looking at faces.5 In contrast, stimulations of color- and face-selective sites evoke notable hallucinations independent of the object being viewed.6 These anecdotal observations suggest that stimulation of high-level visual cortex can evoke perception of complex visual features, but these effects depend on the availability and content of visual input. In this study, we introduce a novel psychophysical task to systematically investigate characteristics of the perceptual events evoked by optogenetic stimulation of macaque inferior temporal (IT) cortex. We trained macaque monkeys to detect and report optogenetic impulses delivered to their IT cortices7,8,9 while holding fixation on object images. In a series of experiments, we show that detection of cortical stimulation is highly dependent on the choice of images presented to the eyes and it is most difficult when fixating on a blank screen. These findings suggest that optogenetic stimulation of high-level visual cortex results in easily detectable distortions of the concurrent contents of vision.

Early stage of radiological expertise modulates resting-state local coherence in the inferior temporal lobe

Background

The visual system and its inherent functions undergo experience-dependent changes through the lifespan, enabling acquisition of new skills. Previous fMRI studies using tasks reported increased specialization in a number of cortical regions subserving visual expertise. Although ample studies focused on representation of long-term visual expertise in the brain, i.e. in terms of year, monthly-based early-stage representation of visual expertise remains unstudied. Given that spontaneous neuronal oscillations actively encode previous experience, we propose brain representations in the resting state is fundamentally important.

Objective

The current study aimed to investigate how monthly-based early-stage visual expertise are represented in the resting state using the expertise model of radiologists.

Methods

In particular, we investigated the altered local clustering pattern of spontaneous brain activity using regional homogeneity (ReHo). A cohort group of radiology interns (n = 22) after one-month training in X-ray department and matched laypersons (n = 22) were recruited after rigorous behavioral assessment.

Results

The results showed higher ReHo in the right hippocampus (HIP) and the right ventral anterior temporal lobe (vATL) (corrected by Alphasim correction, P < 0.05). Moreover, ReHo in the right HIP correlated with the number of cases reviewed during intern radiologists' training (corrected by Alphasim correction, P < 0.05).

Conclusions

In sum, our results demonstrated that the early stage of visual expertise is more concerned with stabilizing visual feature and domain-specific knowledge into long-term memory. The results provided novel evidence regarding how early-stage visual expertise is represented in the resting brain, which help further elaborate how human visual expertise is acquired. We propose that our current study may provide novel ideas for developing new training protocols in medical schools.

A Computational Probe into the Behavioral and Neural Markers of Atypical Facial Emotion Processing in Autism

Despite ample behavioral evidence of atypical facial emotion processing in individuals with autism spectrum disorder (ASD), the neural underpinnings of such behavioral heterogeneities remain unclear. Here, I have used brain-tissue mapped artificial neural network (ANN) models of primate vision to probe candidate neural and behavior markers of atypical facial emotion recognition in ASD at an image-by-image level. Interestingly, the image-level behavioral patterns of the ANNs better matched the neurotypical subjects 'behavior than those measured in ASD. This behavioral mismatch was most remarkable when the ANN behavior was decoded from units that correspond to the primate inferior temporal (IT) cortex. ANN-IT responses also explained a significant fraction of the image-level behavioral predictivity associated with neural activity in the human amygdala (from epileptic patients without ASD), strongly suggesting that the previously reported facial emotion intensity encodes in the human amygdala could be primarily driven by projections from the IT cortex. In sum, these results identify primate IT activity as a candidate neural marker and demonstrate how ANN models of vision can be used to generate neural circuit-level hypotheses and guide future human and nonhuman primate studies in autism.SIGNIFICANCE STATEMENT Moving beyond standard parametric approaches that predict behavior with high-level categorical descriptors of a stimulus (e.g., level of happiness/fear in a face image), in this study, I demonstrate how an image-level probe, using current deep-learning-based ANN models, allows identification of more diagnostic stimuli for autism spectrum disorder enabling the design of more powerful experiments. This study predicts that IT cortex activity is a key candidate neural marker of atypical facial emotion processing in people with ASD. Importantly, the results strongly suggest that ASD-related atypical facial emotion intensity encodes in the human amygdala could be primarily driven by projections from the IT cortex.

It is not just the category: behavioral effects of fMRI-guided electrical microstimulation result from a complex interplay of factors

Functional imaging and electrophysiological studies in primates revealed the existence of patches selective for visual categories in the inferior temporal cortex. Understanding the contribution of these patches to perception requires causal techniques that assess the effect of neural activity manipulations on perception. We used electrical microstimulation (EM) to determine the role of body patch activity in visual categorization in macaques. We tested the hypothesis that EM in a body patch would affect the categorization of bodies versus objects but not of other visual categories. We employed low-current EM of an anterior body patch (ASB) in the superior temporal sulcus, which was defined by functional magnetic resonance imaging and verified with electrophysiological recordings in each session. EM of ASB affected body categorization, but the EM effects were more complex than the expected increase of body-related choices: EM affected the categorization of both body and inanimate images and showed interaction with the choice target location, but its effect was location-specific (tested in 1 subject) on a millimeter scale. Our findings suggest that the behavioral effects of EM in a category-selective patch are not merely a manifestation of the category selectivity of the underlying neuronal population but reflect a complex interplay of multiple factors.

Stimulation of the ventral tegmental area induces visual cortical plasticity at the neuronal level

fMRI studies have shown that pairing a task-irrelevant visual feature with electrical micro-stimulation of the ventral tegmental area (VTA-EM) is sufficient to increase the sensory cortical representation of the paired feature and to improve perceptual performance. However, since fMRI provides an indirect measure of neural activity, the neural response changes underlying the fMRI activations are unknown. Here, we pair a task-irrelevant grating orientation with VTA-EM while attention is directed to a difficult orthogonal task. We examine the changes in neural response properties in macaques by recording spiking activity in the posterior inferior temporal cortex, the locus of fMRI-defined plasticity in previous studies. We observe a relative increase in mean spike rate and preference for the VTA-EM paired orientation compared to an unpaired orientation, which is unrelated to attention. These results demonstrate that VTA-EM-stimulus pairing is sufficient to induce sensory cortical plasticity at the spiking level in nonhuman primates.

Contributions of the Monkey Inferior Temporal Areas TE and TEO to Visual Categorization

The ability to categorize images is thought to depend on neural processing within the ventral visual stream. Recently, we reported that after removal of architectonic area TE, the terminal region of the ventral stream, monkeys were still able to categorize images as cats or dogs moderately well. Here, we investigate the contribution of TEO, the architectonically defined region located one step earlier than area TE in the ventral stream. Bilateral removal of TEO caused only a mild impairment in categorization. However, combined TE + TEO removal was followed by a severe, long-lasting impairment in categorization. All of the monkeys tested, including those with combined TE + TEO removals, had normal low-level visual functions, such as visual acuity. These results support the conclusion that categorization based on visual similarity is processed in parallel in TE and TEO.

Unsupervised changes in core object recognition behavior are predicted by neural plasticity in inferior temporal cortex

Temporal continuity of object identity is a feature of natural visual input and is potentially exploited - in an unsupervised manner - by the ventral visual stream to build the neural representation in inferior temporal (IT) cortex. Here, we investigated whether plasticity of individual IT neurons underlies human core object recognition behavioral changes induced with unsupervised visual experience. We built a single-neuron plasticity model combined with a previously established IT population-to-recognition-behavior-linking model to predict human learning effects. We found that our model, after constrained by neurophysiological data, largely predicted the mean direction, magnitude, and time course of human performance changes. We also found a previously unreported dependency of the observed human performance change on the initial task difficulty. This result adds support to the hypothesis that tolerant core object recognition in human and non-human primates is instructed - at least in part - by naturally occurring unsupervised temporal contiguity experience.

Color Tuning of Face-Selective Neurons in Macaque Inferior Temporal Cortex

What role does color play in the neural representation of complex shapes? We approached the question by measuring color responses of face-selective neurons, using fMRI-guided microelectrode recording of the middle and anterior face patches of inferior temporal cortex (IT) in rhesus macaques. Face-selective cells responded weakly to pure color (equiluminant) photographs of faces. But many of the cells nonetheless showed a bias for warm colors when assessed using images that preserved the luminance contrast relationships of the original photographs. This bias was also found for non-face-selective neurons. Fourier analysis uncovered two components: the first harmonic, accounting for most of the tuning, was biased toward reddish colors, corresponding to the L>M pole of the L-M cardinal axis. The second harmonic showed a bias for modulation between blue and yellow colors axis, corresponding to the S-cone axis. To test what role face-selective cells play in behavior, we related the information content of the neural population with the distribution of face colors. The analyses show that face-selective cells are not optimally tuned to discriminate face colors, but are consistent with the idea that face-selective cells contribute selectively to processing the green-red contrast of faces. The research supports the hypothesis that color-specific information related to the discrimination of objects, including faces, is handled by neural circuits that are independent of shape-selective cortex, as captured by the multistage parallel processing framework of IT (Lafer-Sousa and Conway, 2013).

Fast Recurrent Processing via Ventrolateral Prefrontal Cortex Is Needed by the Primate Ventral Stream for Robust Core Visual Object Recognition

Distributed neural population spiking patterns in macaque inferior temporal (IT) cortex that support core object recognition require additional time to develop for specific, "late-solved" images. This suggests the necessity of recurrent processing in these computations. Which brain circuits are responsible for computing and transmitting these putative recurrent signals to IT? To test whether the ventrolateral prefrontal cortex (vlPFC) is a critical recurrent node in this system, here, we pharmacologically inactivated parts of vlPFC and simultaneously measured IT activity while monkeys performed object discrimination tasks. vlPFC inactivation deteriorated the quality of late-phase (>150 ms from image onset) IT population code and produced commensurate behavioral deficits for late-solved images. Finally, silencing vlPFC caused the monkeys' IT activity and behavior to become more like those produced by feedforward-only ventral stream models. Together with prior work, these results implicate fast recurrent processing through vlPFC as critical to producing behaviorally sufficient object representations in IT.

Expectancy Violation Drives Memory Boost for Stressful Events

Stressful events are often vividly remembered. Although generally adaptive to survival, this emotional-memory enhancement may contribute to stress-related disorders. We tested here whether the enhanced memory for stressful events is due to the expectancy violation evoked by these events. Ninety-four men and women underwent a stressful or control episode. Critically, to manipulate the degree of expectancy violation, we gave participants either detailed or minimal information about the stressor. Although the subjective and hormonal stress responses were comparable in informed and uninformed participants, prior information about the stressor abolished the memory advantage for core features of the stressful event, tested 7 days later. Using functional near-infrared spectroscopy, we further linked the expectancy violation and memory formation under stress to the inferior temporal cortex. These data are the first to show that detailed information about an upcoming stressor and, by implication, a reduced expectancy violation attenuates the memory for stressful events.

Distinct mechanisms of face representation by enhancive and suppressive neurons of the inferior temporal cortex

Face-selective neurons in the inferior temporal (IT) cortex respond to faces by either increasing (ENH) or decreasing (SUP) their spiking activities compared with their baseline. Although nearly half of IT face neurons are selectively suppressed by face stimulation, their role in face representation is not clear. To address this issue, we recorded the spiking activities and local field potential (LFP) from IT cortex of three monkeys while they viewed a large set of visual stimuli. LFP high-gamma (HG-LFP) power indicated the presence of both ENH and SUP face-selective neural clusters in IT cortex. The magnitude of HG-LFP power of the recording sites was correlated with the magnitude of change in the evoked spiking activities of its constituent neurons for both ENH and SUP face clusters. Spatial distribution of the ENH and SUP face clusters suggests the presence of a complex and heterogeneous face hypercluster organization in IT cortex. Importantly, ENH neurons conveyed more face category and SUP neurons conveyed more face identity information at both the single-unit and neuronal population levels. Onset and peak of suppressive single-unit, neuronal population, and HG-LFP power activities lagged those of the ENH ones. These results demonstrate that IT neuronal code for face representation is optimized by increasing sparseness through selective suppression of a subset of face neurons. We suggest that IT cortex contains spatial clusters of both ENH and SUP face neurons with distinct specialized functional role in face representation.NEW & NOTEWORTHY Electrophysiological and imaging studies have suggested that face information is encoded by a network of clusters of enhancive face-selective neurons in the visual cortex of man and monkey. We show that nearly half of face-selective neurons are suppressed by face stimulation. The suppressive neurons form spatial clusters and convey more face identity information than the enhancive face neurons. Our results suggest the presence of two neuronal subsystems for coarse and fine face information processing.

Clustering of activated microglia occurs before the formation of dystrophic neurites in the evolution of Aβ plaques in Alzheimer's disease

Alzheimer's disease (AD) is a late-onset disease that has proved difficult to model. Microglia are implicated in AD, but reports vary on precisely when and how in the sequence of pathological changes they become involved. Here, post-mortem human tissue from two differentially affected regions of the AD brain and from non-demented individuals with a high load of AD-type pathology (high pathology controls) was used to model the disease time course in order to determine how microglial activation relates temporally to the deposition of hallmark amyloid-β (Aβ) and hyperphosphorylated microtubule associated protein tau pathology. Immunofluorescence against the pan-microglial marker, ionised calcium-binding adapter molecule 1 (IBA1), Aβ and tau, was performed in the primary motor cortex (PMC), a region relatively spared of AD pathological changes, and compared to the severely affected inferior temporal cortex (ITC) in the same cases. Unlike the ITC, the PMC in the AD cases was spared of any degenerative changes in cortical thickness and the density of Betz cells and total neurons. The clustering of activated microglia was greatest in the PMC of AD cases and high pathology controls compared to the ITC. This suggests microglial activation is most prominent in the early phases of AD pathophysiology. Nascent tau inclusions were found in neuritic plaques in the PMC but were more numerous in the ITC of the same case. This shows that tau positive neuritic plaques begin early in AD which is likely of pathogenic importance, however major tau deposition follows the accumulation of Aβ and clustering of activated microglia. Importantly, findings presented here demonstrate that different states of microglial activation, corresponding to regional accumulations of Aβ and tau, are present simultaneously in the same individual; an important factor for consideration if targeting these cells for therapeutic intervention.

Reversible Inactivation of Different Millimeter-Scale Regions of Primate IT Results in Different Patterns of Core Object Recognition Deficits

Extensive research suggests that the inferior temporal (IT) population supports visual object recognition behavior. However, causal evidence for this hypothesis has been equivocal, particularly beyond the specific case of face-selective subregions of IT. Here, we directly tested this hypothesis by pharmacologically inactivating individual, millimeter-scale subregions of IT while monkeys performed several core object recognition subtasks, interleaved trial-by trial. First, we observed that IT inactivation resulted in reliable contralateral-biased subtask-selective behavioral deficits. Moreover, inactivating different IT subregions resulted in different patterns of subtask deficits, predicted by each subregion's neuronal object discriminability. Finally, the similarity between different inactivation effects was tightly related to the anatomical distance between corresponding inactivation sites. Taken together, these results provide direct evidence that the IT cortex causally supports general core object recognition and that the underlying IT coding dimensions are topographically organized.

The Organization and Operation of Inferior Temporal Cortex

Inferior temporal cortex (IT) is a key part of the ventral visual pathway implicated in object, face, and scene perception. But how does IT work? Here, I describe an organizational scheme that marries form and function and provides a framework for future research. The scheme consists of a series of stages arranged along the posterior-anterior axis of IT, defined by anatomical connections and functional responses. Each stage comprises a complement of subregions that have a systematic spatial relationship. The organization of each stage is governed by an eccentricity template, and corresponding eccentricity representations across stages are interconnected. Foveal representations take on a role in high-acuity object vision (including face recognition); intermediate representations compute other aspects of object vision such as behavioral valence (using color and surface cues); and peripheral representations encode information about scenes. This multistage, parallel-processing model invokes an innately determined organization refined by visual experience that is consistent with principles of cortical development. The model is also consistent with principles of evolution, which suggest that visual cortex expanded through replication of retinotopic areas. Finally, the model predicts that the most extensively studied network within IT-the face patches-is not unique but rather one manifestation of a canonical set of operations that reveal general principles of how IT works.

Face Repetition Probability Does Not Affect Repetition Suppression in Macaque Inferotemporal Cortex

Repetition suppression, which refers to reduced neural activity for repeated stimuli, is typically explained by bottom-up or local adaptation mechanisms. However, recent theories have emphasized the role of top-down processes, suggesting that this response reduction reflects the fulfillment of perceptual expectations. To support this, an influential human fMRI study showed that the magnitude of suppression is modulated by the probability of a repetition. No such repetition probability effect was found in macaque inferior temporal (IT) cortex for spiking activity despite the presence of repetition suppression. Contrary to the human fMRI studies that showed an effect of repetition probability, the macaque single-unit study used a large variety of unfamiliar stimuli and the monkeys were not required to attend the stimuli. Here, as in the human fMRI studies, we used faces as stimuli and made the monkeys attend to the stimulus content. We simultaneously recorded spiking activity and local field potentials (LFPs) in the middle lateral face patch (ML) of one monkey (male) and a face-responsive region of another (female). Although we observed significant repetition suppression of spiking activity and high gamma-band LFPs in both animals, there were no effects of repetition probability even when repetitions were task relevant and repetition probability affected behavioral decisions. In conclusion, despite the use of face stimuli and a stimulus-related task, no neural signature of repetition probability was present for faces in a face responsive patch of macaque IT. This further challenges a general perceptual expectation account of repetition suppression.SIGNIFICANCE STATEMENT Repetition suppression is a reduced brain activity for repeated stimuli commonly observed across species. In the predictive coding framework, such suppression is thought to reflect fulfilled perceptual expectations. Although this hypothesis is supported by several human fMRI studies reporting an effect of repetition probability on repetition suppression, this could not be replicated in single-cell recordings in monkey inferior temporal (IT) cortex. Subsequent studies narrowed down the conditions for the effect to requiring attention and being limited to particular stimulus categories such as faces. Here, we show that, even under these conditions, repetition suppression in monkey IT neurons is still unaffected by repetition probability, even in a task with a behavioral effect, challenging the perceptual expectation account of repetition suppression.

Attractor Dynamics in Networks with Learning Rules Inferred from In Vivo Data

The attractor neural network scenario is a popular scenario for memory storage in the association cortex, but there is still a large gap between models based on this scenario and experimental data. We study a recurrent network model in which both learning rules and distribution of stored patterns are inferred from distributions of visual responses for novel and familiar images in the inferior temporal cortex (ITC). Unlike classical attractor neural network models, our model exhibits graded activity in retrieval states, with distributions of firing rates that are close to lognormal. Inferred learning rules are close to maximizing the number of stored patterns within a family of unsupervised Hebbian learning rules, suggesting that learning rules in ITC are optimized to store a large number of attractor states. Finally, we show that there exist two types of retrieval states: one in which firing rates are constant in time and another in which firing rates fluctuate chaotically.

Mechanisms for shaping receptive field in monkey area TE

Visual object information is conveyed from V1 to area TE along the ventral visual pathway with increasing receptive field (RF) sizes. The RFs of TE neurons are known to be large, but it is largely unknown how large RFs are shaped along the ventral visual pathway. In this study, we addressed this question in two aspects, static and dynamic mechanisms, by recording neural responses from macaque area TE and V4 to object stimuli presented at various locations in the visual field. As a component related to static mechanisms, we found that in area TE, but not in V4, response latency to objects presented at fovea were different from objects in periphery. As a component of the dynamic mechanisms, we examined effects of spatial attention on the RFs of TE neurons. Spatial attention did not affect response latency but modulated response magnitudes depending on attended location, shifting of the longitudinal axis of RFs toward the attended locations. In standard models of large RF formation, downstream neurons pool information from nearby RFs, and this process is repeated across the visual field and at each step along the ventral visual pathway. The present study revealed that this mechanism is not that simple: 1) different circuit mechanisms for foveal and peripheral visual fields may be situated between V4 and area TE, and 2) spatial attention dynamically changes the shape of RFs.NEW & NOTEWORTHY Receptive fields (RFs) of neurons are progressively increased along the ventral visual pathway so that an RF at the final stage, area TE, covers a large area of the visual field. We explored the mechanism and suggested involvement of parallel circuit mechanisms between V4 and TE for foveal and peripheral parts of visual field. We also found a dynamic component of RF shape formation through attentional modulation of responses in a location-dependent manner.

Phase-Amplitude Coupling and Long-Range Phase Synchronization Reveal Frontotemporal Interactions during Visual Working Memory

It has been suggested that cross-frequency phase-amplitude coupling (PAC), particularly in temporal brain structures, serves as a neural mechanism for coordinated working memory storage. In this magnetoencephalography study, we show that during visual working memory maintenance, temporal cortex regions, which exhibit enhanced PAC, interact with prefrontal cortex via enhanced low-frequency phase synchronization. Healthy human participants were engaged in a visual delayed match-to-sample task with pictures of natural objects. During the delay period, we observed increased spectral power of beta (20-28 Hz) and gamma (40-94 Hz) bands as well as decreased power of theta/alpha band (7-9 Hz) oscillations in visual sensory areas. Enhanced PAC between the phases of theta/alpha and the amplitudes of beta oscillations was found in the left inferior temporal cortex (IT), an area known to be involved in visual object memory. Furthermore, the IT was functionally connected to the prefrontal cortex by increased low-frequency phase synchronization within the theta/alpha band. Together, these results point to a mechanism in which the combination of PAC and long-range phase synchronization subserves enhanced large-scale brain communication. They suggest that distant brain regions might coordinate their activity in the low-frequency range to engage local stimulus-related processing in higher frequencies via the combination of long-range, within-frequency phase synchronization and local cross-frequency PAC.

SIGNIFICANCE STATEMENT

Working memory maintenance, like other cognitive functions, requires the coordinated engagement of brain areas in local and large-scale networks. However, the mechanisms by which spatially distributed brain regions share and combine information remain primarily unknown. We show that the combination of long-range, low-frequency phase synchronization and local cross-frequency phase-amplitude coupling might serve as a mechanism to coordinate memory processes across distant brain areas. In this study, low-frequency phase synchronization between prefrontal and temporal cortex co-occurred with local cross-frequency phase-amplitude coupling to higher frequencies in the latter. By such means, ongoing working memory storage taking place in higher frequencies in temporal regions might be effectively coordinated by distant frontal brain regions through synchronized activity in the low-frequency range.

The Code for Facial Identity in the Primate Brain

Primates recognize complex objects such as faces with remarkable speed and reliability. Here, we reveal the brain's code for facial identity. Experiments in macaques demonstrate an extraordinarily simple transformation between faces and responses of cells in face patches. By formatting faces as points in a high-dimensional linear space, we discovered that each face cell's firing rate is proportional to the projection of an incoming face stimulus onto a single axis in this space, allowing a face cell ensemble to encode the location of any face in the space. Using this code, we could precisely decode faces from neural population responses and predict neural firing rates to faces. Furthermore, this code disavows the long-standing assumption that face cells encode specific facial identities, confirmed by engineering faces with drastically different appearance that elicited identical responses in single face cells. Our work suggests that other objects could be encoded by analogous metric coordinate systems. PAPERCLIP.

Divisive Normalization Predicts Adaptation-Induced Response Changes in Macaque Inferior Temporal Cortex

Stimulus repetition alters neural responses to the repeated stimulus. This so-called adaptation phenomenon has been commonly observed at multiple spatial and temporal scales and in different brain areas, and has been hypothesized to affect the neural representation of the sensory input. Yet, the neural mechanisms underlying adaptation still remain unclear, especially in higher-order cortical areas. Here we employ a divisive normalization model of neural responses to predict adaptation-induced changes in responses of single neurons in the macaque inferior temporal (IT) cortex. According to this model, the response of a neuron is determined by an interplay between its direct excitatory and divisive normalizing inputs, with each input being subject to adaptation. To test the model, we recorded the responses of single IT cortex neurons to complex visual stimuli while separately adapting the two putative types of input to those neurons. We compared the changes in responses of these neurons following such adaptation with predictions derived from the divisive normalization model. As predicted by the model, we show that adaptation in the IT cortex can, depending on the relative strength of each putative type of input to a neuron, suppress or enhance the neural response to a complex stimulus. More generally, our data suggest that adaptation serves to selectively enhance processing of the stimuli that differ from recently experienced ones, even when these occur within a configuration of multiple stimuli.

SIGNIFICANCE STATEMENT

Stimulus repetition alters neural responses to the repeated stimulus. This so-called adaptation phenomenon has been robustly demonstrated in brains of different species and is considered to be a form of short-term plasticity inherent to the processing of sensory stimuli. Nevertheless, the functional role and underlying mechanisms of adaptation remain unclear. Here we demonstrate that divisive normalization, a canonical neural computation operating throughout the brain, predicts the adaptation-induced changes in response of single neurons to complex stimulus configurations in the macaque inferotemporal cortex. Our findings embed adaptation effects of inferotemporal neurons into the context of a broader neural network perspective that includes divisive normalization. Additionally, our findings have implications for understanding of the function of adaptation in higher-order sensory cortices.

Shape Selectivity of Middle Superior Temporal Sulcus Body Patch Neurons

Functional MRI studies in primates have demonstrated cortical regions that are strongly activated by visual images of bodies. The presence of such body patches in macaques allows characterization of the stimulus selectivity of their single neurons. Middle superior temporal sulcus body (MSB) patch neurons showed similar stimulus selectivity for natural, shaded, and textured images compared with their silhouettes, suggesting that shape is an important determinant of MSB responses. Here, we examined and modeled the shape selectivity of single MSB neurons. We measured the responses of single MSB neurons to a variety of shapes producing a wide range of responses. We used an adaptive stimulus sampling procedure, selecting and modifying shapes based on the responses of the neuron. Forty percent of shapes that produced the maximal response were rated by humans as animal-like, but the top shape of many MSB neurons was not judged as resembling a body. We fitted the shape selectivity of MSB neurons with a model that parameterizes shapes in terms of curvature and orientation of contour segments, with a pixel-based model, and with layers of units of convolutional neural networks (CNNs). The deep convolutional layers of CNNs provided the best goodness-of-fit, with a median explained explainable variance of the neurons’ responses of 77%. The goodness-of-fit increased along the convolutional layers’ hierarchy but was lower for the fully connected layers. Together with demonstrating the successful modeling of single unit shape selectivity with deep CNNs, the data suggest that semantic or category knowledge determines only slightly the single MSB neuron’s shape selectivity.

Postnatal Dendritic Growth and Spinogenesis of Layer-V Pyramidal Cells Differ between Visual, Inferotemporal, and Prefrontal Cortex of the Macaque Monkey

Pyramidal cells in the primate cerebral cortex, particularly those in layer III, exhibit regional variation in both the time course and magnitude of postnatal growth and pruning of dendrites and spines. Less is known about the development of pyramidal cell dendrites and spines in other cortical layers. Here we studied dendritic morphology of layer-V pyramidal cells in primary visual cortex (V1, sensory), cytoarchitectonic area TE in the inferotemporal cortex (sensory association), and granular prefrontal cortex (Walker's area 12, executive) of macaque monkeys at the ages of 2 days, 3 weeks, 3.5 months, and 4.5 years. We found that changes in the basal dendritic field area of pyramidal cells were different across the three areas. In V1, field size became smaller over time (largest at 2 days, half that size at 4.5 years), in TE it did not change, and in area 12 it became larger over time (smallest at 2 days, 1.5 times greater at 4.5 years). In V1 and TE, the total number of branch points in the basal dendritic trees was similar between 2 days and 4.5 years, while in area 12 the number was greater in the adult monkeys than in the younger ones. Spine density peaked at 3 weeks and declined in all areas by adulthood, with V1 exhibiting a faster decline than area TE or area 12. Estimates of the total number of spines in the dendritic trees revealed that following the onset of visual experience, pyramidal cells in V1 lose more spines than they grow, whereas those in TE and area 12 grow more spines than they lose during the same period. These data provide further evidence that the process of synaptic refinement in cortical pyramidal cells differs not only according to time, but also location within the cortex. Furthermore, given the previous finding that layer-III pyramidal cells in all these areas exhibit the highest density and total number of spines at 3.5 months, the current results indicate that pyramidal cells in layers III and V develop spines at different rates.

Information Accumulation over Time in Monkey Inferior Temporal Cortex Neurons Explains Pattern Recognition Reaction Time under Visual Noise

We recognize objects even when they are partially degraded by visual noise. We studied the relation between the amount of visual noise (5, 10, 15, 20, or 25%) degrading 8 black-and-white stimuli and stimulus identification in 2 monkeys performing a sequential delayed match-to-sample task. We measured the accuracy and speed with which matching stimuli were identified. The performance decreased slightly (errors increased) as the amount of visual noise increased for both monkeys. The performance remained above 80% correct, even with 25% noise. However, the reaction times markedly increased as the noise increased, indicating that the monkeys took progressively longer to decide what the correct response would be as the amount of visual noise increased, showing that the monkeys trade time to maintain accuracy. Thus, as time unfolds the monkeys act as if they are accumulating the information and/or testing hypotheses about whether the test stimulus is likely to be a match for the sample being held in short-term memory. We recorded responses from 13 single neurons in area TE of the 2 monkeys. We found that stimulus-selective information in the neuronal responses began accumulating when the match stimulus appeared. We found that the greater the amount of noise obscuring the test stimulus, the more slowly stimulus-related information by the 13 neurons accumulated. The noise induced slowing was about the same for both behavior and information. These data are consistent with the hypothesis that area TE neuron population carries information about stimulus identity that accumulates over time in such a manner that it progressively overcomes the signal degradation imposed by adding visual noise.

Neurophysiological Organization of the Middle Face Patch in Macaque Inferior Temporal Cortex

While early cortical visual areas contain fine scale spatial organization of neuronal properties, such as orientation preference, the spatial organization of higher-level visual areas is less well understood. The fMRI demonstration of face-preferring regions in human ventral cortex and monkey inferior temporal cortex ("face patches") raises the question of how neural selectivity for faces is organized. Here, we targeted hundreds of spatially registered neural recordings to the largest fMRI-identified face-preferring region in monkeys, the middle face patch (MFP), and show that the MFP contains a graded enrichment of face-preferring neurons. At its center, as much as 93% of the sites we sampled responded twice as strongly to faces than to nonface objects. We estimate the maximum neurophysiological size of the MFP to be ∼6 mm in diameter, consistent with its previously reported size under fMRI. Importantly, face selectivity in the MFP varied strongly even between neighboring sites. Additionally, extremely face-selective sites were ∼40 times more likely to be present inside the MFP than outside. These results provide the first direct quantification of the size and neural composition of the MFP by showing that the cortical tissue localized to the fMRI defined region consists of a very high fraction of face-preferring sites near its center, and a monotonic decrease in that fraction along any radial spatial axis.

SIGNIFICANCE STATEMENT

The underlying organization of neurons that give rise to the large spatial regions of activity observed with fMRI is not well understood. Neurophysiological studies that have targeted the fMRI identified face patches in monkeys have provided evidence for both large-scale clustering and a heterogeneous spatial organization. Here we used a novel x-ray imaging system to spatially map the responses of hundreds of sites in and around the middle face patch. We observed that face-selective signal localized to the middle face patch was characterized by a gradual spatial enrichment. Furthermore, strongly face-selective sites were ∼40 times more likely to be found inside the patch than outside of the patch.

Seeing a straight line on a curved surface: decoupling of patterns from surfaces by single IT neurons

We have no difficulty seeing a straight line on a curved piece of paper, but in fact, doing so requires decoupling the shape of the surface from the pattern itself. Here we report a novel form of invariance in the visual cortex: single neurons in monkey inferior temporal cortex respond similarly to congruent transformations of patterns and surfaces, in effect decoupling patterns from the surface on which they are overlaid.

We have no difficulty seeing a straight line drawn on a paper even when the paper is bent, but this inference is in fact nontrivial. Doing so requires either matching local features or representing the pattern after factoring out the surface shape. Here we show that single neurons in the monkey inferior temporal (IT) cortex show invariant responses to patterns across rigid and nonrigid changes of surfaces. We recorded neuronal responses to stimuli in which the pattern and the surrounding surface were varied independently. In a subset of neurons, we found pattern-surface interactions that produced similar responses to stimuli across congruent pattern and surface transformations. These interactions produced systematic shifts in curvature tuning of patterns when overlaid on convex and flat surfaces. Our results show that surfaces are factored out of patterns by single neurons, thereby enabling complex perceptual inferences.

NEW & NOTEWORTHY We have no difficulty seeing a straight line on a curved piece of paper, but in fact, doing so requires decoupling the shape of the surface from the pattern itself. Here we report a novel form of invariance in the visual cortex: single neurons in monkey inferior temporal cortex respond similarly to congruent transformations of patterns and surfaces, in effect decoupling patterns from the surface on which they are overlaid.

Temporal dynamics of visual category representation in the macaque inferior temporal cortex

Object categories are recognized at multiple levels of hierarchical abstractions. Psychophysical studies have shown a more rapid perceptual access to the mid-level category information (e.g., human faces) than the higher (superordinate; e.g., animal) or the lower (subordinate; e.g., face identity) level. Mid-level category members share many features, whereas few features are shared among members of different mid-level categories. To understand better the neural basis of expedited access to mid-level category information, we examined neural responses of the inferior temporal (IT) cortex of macaque monkeys viewing a large number of object images. We found an earlier representation of mid-level categories in the IT population and single-unit responses compared with superordinate- and subordinate-level categories. The short-latency representation of mid-level category information shows that visual cortex first divides the category shape space at its sharpest boundaries, defined by high/low within/between-group similarity. This short-latency, mid-level category boundary map may be a prerequisite for representation of other categories at more global and finer scales.

Stimulus features coded by single neurons of a macaque body category selective patch

Body category-selective regions of the primate temporal cortex respond to images of bodies, but it is unclear which fragments of such images drive single neurons' responses in these regions. Here we applied the Bubbles technique to the responses of single macaque middle superior temporal sulcus (midSTS) body patch neurons to reveal the image fragments the neurons respond to. We found that local image fragments such as extremities (limbs), curved boundaries, and parts of the torso drove the large majority of neurons. Bubbles revealed the whole body in only a few neurons. Neurons coded the features in a manner that was tolerant to translation and scale changes. Most image fragments were excitatory but for a few neurons both inhibitory and excitatory fragments (opponent coding) were present in the same image. The fragments we reveal here in the body patch with Bubbles differ from those suggested in previous studies of face-selective neurons in face patches. Together, our data indicate that the majority of body patch neurons respond to local image fragments that occur frequently, but not exclusively, in bodies, with a coding that is tolerant to translation and scale. Overall, the data suggest that the body category selectivity of the midSTS body patch depends more on the feature statistics of bodies (e.g., extensions occur more frequently in bodies) than on semantics (bodies as an abstract category).

Repetition suppression for visual actions in the macaque superior temporal sulcus

In many brain areas, repetition of a stimulus usually weakens the neural response. This "adaptation" or repetition suppression effect has been observed with mass potential measures such as event-related potentials (ERPs), in fMRI BOLD responses, and locally with local field potentials (LFPs) and spiking activity. Recently, it has been reported that macaque F5 mirror neurons do not show repetition suppression of their spiking activity for single repetitions of hand actions, which disagrees with human fMRI adaptation studies. This finding also contrasts with numerous studies showing repetition suppression in macaque inferior temporal cortex, including the rostral superior temporal sulcus (STS). Since the latter studies employed static stimuli, we assessed here whether the use of dynamic action stimuli abolishes repetition suppression in the awake macaque STS. To assess adaptation effects in the STS, we employed the same hand action movies as used when examining adaptation in F5. The upper bank STS neurons showed repetition suppression during the approaching phase of the hand action, which corresponded to the phase of the action for which these neurons responded overall the strongest. The repetition suppression was present for the spiking activity measured in independent single-unit and multiunit recordings as well as for the LFP power at frequencies > 50 Hz. Together with previous data in F5, these findings suggest that adaptation effects differ between F5 mirror neurons and the STS neurons.

Optogenetic and pharmacological suppression of spatial clusters of face neurons reveal their causal role in face gender discrimination

Neurons that respond more to images of faces over nonface objects were identified in the inferior temporal (IT) cortex of primates three decades ago. Although it is hypothesized that perceptual discrimination between faces depends on the neural activity of IT subregions enriched with "face neurons," such a causal link has not been directly established. Here, using optogenetic and pharmacological methods, we reversibly suppressed the neural activity in small subregions of IT cortex of macaque monkeys performing a facial gender-discrimination task. Each type of intervention independently demonstrated that suppression of IT subregions enriched in face neurons induced a contralateral deficit in face gender-discrimination behavior. The same neural suppression of other IT subregions produced no detectable change in behavior. These results establish a causal link between the neural activity in IT face neuron subregions and face gender-discrimination behavior. Also, the demonstration that brief neural suppression of specific spatial subregions of IT induces behavioral effects opens the door for applying the technical advantages of optogenetics to a systematic attack on the causal relationship between IT cortex and high-level visual perception.

Dynamics of 3D view invariance in monkey inferotemporal cortex

Rotations in depth are challenging for object vision because features can appear, disappear, be stretched or compressed. Yet we easily recognize objects across views. Are the underlying representations view invariant or dependent? This question has been intensely debated in human vision, but the neuronal representations remain poorly understood. Here, we show that for naturalistic objects, neurons in the monkey inferotemporal (IT) cortex undergo a dynamic transition in time, whereby they are initially sensitive to viewpoint and later encode view-invariant object identity. This transition depended on two aspects of object structure: it was strongest when objects foreshortened strongly across views and were similar to each other. View invariance in IT neurons was present even when objects were reduced to silhouettes, suggesting that it can arise through similarity between external contours of objects across views. Our results elucidate the viewpoint debate by showing that view invariance arises dynamically in IT neurons out of a representation that is initially view dependent.

Correlated activity supports efficient cortical processing

Visual recognition is a computational challenge that is thought to occur via efficient coding. An important concept is sparseness, a measure of coding efficiency. The prevailing view is that sparseness supports efficiency by minimizing redundancy and correlations in spiking populations. Yet, we recently reported that "choristers", neurons that behave more similarly (have correlated stimulus preferences and spontaneous coincident spiking), carry more generalizable object information than uncorrelated neurons ("soloists") in macaque inferior temporal (IT) cortex. The rarity of choristers (as low as 6% of IT neurons) indicates that they were likely missed in previous studies. Here, we report that correlation strength is distinct from sparseness (choristers are not simply broadly tuned neurons), that choristers are located in non-granular output layers, and that correlated activity predicts human visual search efficiency. These counterintuitive results suggest that a redundant correlational structure supports efficient processing and behavior.

Task dependence of visual and category representations in prefrontal and inferior temporal cortices

Visual categorization is an essential perceptual and cognitive process for assigning behavioral significance to incoming stimuli. Categorization depends on sensory processing of stimulus features as well as flexible cognitive processing for classifying stimuli according to the current behavioral context. Neurophysiological studies suggest that the prefrontal cortex (PFC) and the inferior temporal cortex (ITC) are involved in visual shape categorization. However, their precise roles in the perceptual and cognitive aspects of the categorization process are unclear, as the two areas have not been directly compared during changing task contexts. To address this, we examined the impact of task relevance on categorization-related activity in PFC and ITC by recording from both areas as monkeys alternated between a shape categorization and passive viewing tasks. As monkeys viewed the same stimuli in both tasks, the impact of task relevance on encoding in each area could be compared. While both areas showed task-dependent modulations of neuronal activity, the patterns of results differed markedly. PFC, but not ITC, neurons showed a modest increase in firing rates when stimuli were task relevant. PFC also showed significantly stronger category selectivity during the task compared with passive viewing, while task-dependent modulations of category selectivity in ITC were weak and occurred with a long latency. Finally, both areas showed an enhancement of stimulus selectivity during the task compared with passive viewing. Together, this suggests that the ITC and PFC show differing degrees of task-dependent flexibility and are preferentially involved in the perceptual and cognitive aspects of the categorization process, respectively.

Neurons in macaque inferior temporal cortex show no surprise response to deviants in visual oddball sequences

Many studies measured neural responses in oddball paradigms, showing a different response to the same stimulus when presented with a low (deviant) compared with a high probability (standard) in a sequence. Such a differential response is manifested in event-related potential studies as the mismatch negativity (MMN) and has been observed in several sensory modalities, including vision. Other studies showed that stimulus repetition suppresses the neural response. It has been suggested that this adaptation effect underlies the smaller responses to the standard compared with the deviant stimulus in oddball sequences. However, the MMN may also reflect the violation of a prediction based on the sequence of standards, i.e., a surprise response. We examined the presence of a surprise response to deviants in visual oddball sequences in macaque (Macaca mulatta) inferior temporal (IT) cortex, a higher-order cortical area. In agreement with visual MMN studies, single-unit IT responses were greater for the deviant than for the standard stimuli. However, single IT neurons showed no greater response to the deviant stimulus in the oddball sequence than to the same stimulus presented with the same probability in a sequence that consisted of many stimuli. LFPs also showed no evidence of a surprise response. These data suggest that stimulus-specific adaptation, without a surprise-related boost of activity to the deviant, underlies the responses in visual oddball sequences even in higher visual cortex. Furthermore, we show that for IT neurons such adaptive mechanisms take into account a relatively short stimulus history, with weaker effects at longer time scales.

High baseline activity in inferior temporal cortex improves neural and behavioral discriminability during visual categorization

Spontaneous firing is a ubiquitous property of neural activity in the brain. Recent literature suggests that this baseline activity plays a key role in perception. However, it is not known how the baseline activity contributes to neural coding and behavior. Here, by recording from the single neurons in the inferior temporal cortex of monkeys performing a visual categorization task, we thoroughly explored the relationship between baseline activity, the evoked response, and behavior. Specifically we found that a low-frequency (<8 Hz) oscillation in the spike train, prior and phase-locked to the stimulus onset, was correlated with increased gamma power and neuronal baseline activity. This enhancement of the baseline activity was then followed by an increase in the neural selectivity and the response reliability and eventually a higher behavioral performance.

Texture discriminability in monkey inferotemporal cortex predicts human texture perception

Shape and texture are both important properties of visual objects, but texture is relatively less understood. Here, we characterized neuronal responses to discrete textures in monkey inferotemporal (IT) cortex and asked whether they can explain classic findings in human texture perception. We focused on three classic findings on texture discrimination: 1) it can be easy or hard depending on the constituent elements; 2) it can have asymmetries, and 3) it is reduced for textures with randomly oriented elements. We recorded neuronal activity from monkey inferotemporal (IT) cortex and measured texture perception in humans for a variety of textures. Our main findings are as follows: 1) IT neurons show congruent selectivity for textures across array size; 2) textures that were easy for humans to discriminate also elicited distinct patterns of neuronal activity in monkey IT; 3) texture pairs with asymmetries in humans also exhibited asymmetric variation in firing rate across monkey IT; and 4) neuronal responses to randomly oriented textures were explained by an average of responses to homogeneous textures, which rendered them less discriminable. The reduction in discriminability of monkey IT neurons predicted the reduced discriminability in humans during texture discrimination. Taken together, our results suggest that texture perception in humans is likely based on neuronal representations similar to those in monkey IT.

Distinct neuronal interactions in anterior inferotemporal areas of macaque monkeys during retrieval of object association memory

In macaque monkeys, the anterior inferotemporal cortex, a region crucial for object memory processing, is composed of two adjacent, hierarchically distinct areas, TE and 36, for which different functional roles and neuronal responses in object memory tasks have been characterized. However, it remains unknown how the neuronal interactions differ between these areas during memory retrieval. Here, we conducted simultaneous recordings from multiple single-units in each of these areas while monkeys performed an object association memory task and examined the inter-area differences in neuronal interactions during the delay period. Although memory neurons showing sustained activity for the presented cue stimulus, cue-holding (CH) neurons, interacted with each other in both areas, only those neurons in area 36 interacted with another type of memory neurons coding for the to-be-recalled paired associate (pair-recall neurons) during memory retrieval. Furthermore, pairs of CH neurons in area TE showed functional coupling in response to each individual object during memory retention, whereas the same class of neuron pairs in area 36 exhibited a comparable strength of coupling in response to both associated objects. These results suggest predominant neuronal interactions in area 36 during the mnemonic processing, which may underlie the pivotal role of this brain area in both storage and retrieval of object association memory.

The importance of visual features in generic vs. specialized object recognition: a computational study

It is debated whether the representation of objects in inferior temporal (IT) cortex is distributed over activities of many neurons or there are restricted islands of neurons responsive to a specific set of objects. There are lines of evidence demonstrating that fusiform face area (FFA-in human) processes information related to specialized object recognition (here we say within category object recognition such as face identification). Physiological studies have also discovered several patches in monkey ventral temporal lobe that are responsible for facial processing. Neuronal recording from these patches shows that neurons are highly selective for face images whereas for other objects we do not see such selectivity in IT. However, it is also well-supported that objects are encoded through distributed patterns of neural activities that are distinctive for each object category. It seems that visual cortex utilize different mechanisms for between category object recognition (e.g., face vs. non-face objects) vs. within category object recognition (e.g., two different faces). In this study, we address this question with computational simulations. We use two biologically inspired object recognition models and define two experiments which address these issues. The models have a hierarchical structure of several processing layers that simply simulate visual processing from V1 to aIT. We show, through computational modeling, that the difference between these two mechanisms of recognition can underlie the visual feature and extraction mechanism. It is argued that in order to perform generic and specialized object recognition, visual cortex must separate the mechanisms involved in within category from between categories object recognition. High recognition performance in within category object recognition can be guaranteed when class-specific features with intermediate size and complexity are extracted. However, generic object recognition requires a distributed universal dictionary of visual features in which the size of features does not have significant difference.

Behavioral demand modulates object category representation in the inferior temporal cortex

Visual object categorization is a critical task in our daily life. Many studies have explored category representation in the inferior temporal (IT) cortex at the level of single neurons and population. However, it is not clear how behavioral demands modulate this category representation. Here, we recorded from the IT single neurons in monkeys performing two different tasks with identical visual stimuli: passive fixation and body/object categorization. We found that category selectivity of the IT neurons was improved in the categorization compared with the passive task where reward was not contingent on image category. The category improvement was the result of larger rate enhancement for the preferred category and smaller response variability for both preferred and nonpreferred categories. These specific modulations in the responses of IT category neurons enhanced signal-to-noise ratio of the neural responses to discriminate better between the preferred and nonpreferred categories. Our results provide new insight into the adaptable category representation in the IT cortex, which depends on behavioral demands.

Factors modulating neural reactivity to drug cues in addiction: a survey of human neuroimaging studies

Human neuroimaging studies suggest that neural cue reactivity is strongly associated with indices of drug use, including addiction severity and treatment success. However, little is known about factors that modulate cue reactivity. The goal of this review, in which we survey published fMRI and PET studies on drug cue reactivity in cocaine, alcohol, and tobacco cigarette users, is to highlight major factors that modulate brain reactivity to drug cues. First, we describe cue reactivity paradigms used in neuroimaging research and outline the brain circuits that underlie cue reactivity. We then discuss major factors that have been shown to modulate cue reactivity and review specific evidence as well as outstanding questions related to each factor. Building on previous model-building reviews on the topic, we then outline a simplified model that includes the key modulatory factors and a tentative ranking of their relative impact. We conclude with a discussion of outstanding challenges and future research directions, which can inform future neuroimaging studies as well as the design of treatment and prevention programs.

Transformation-invariant visual representations in self-organizing spiking neural networks

The ventral visual pathway achieves object and face recognition by building transformation-invariant representations from elementary visual features. In previous computer simulation studies with rate-coded neural networks, the development of transformation-invariant representations has been demonstrated using either of two biologically plausible learning mechanisms, Trace learning and Continuous Transformation (CT) learning. However, it has not previously been investigated how transformation-invariant representations may be learned in a more biologically accurate spiking neural network. A key issue is how the synaptic connection strengths in such a spiking network might self-organize through Spike-Time Dependent Plasticity (STDP) where the change in synaptic strength is dependent on the relative times of the spikes emitted by the presynaptic and postsynaptic neurons rather than simply correlated activity driving changes in synaptic efficacy. Here we present simulations with conductance-based integrate-and-fire (IF) neurons using a STDP learning rule to address these gaps in our understanding. It is demonstrated that with the appropriate selection of model parameters and training regime, the spiking network model can utilize either Trace-like or CT-like learning mechanisms to achieve transform-invariant representations.