Neuronal responses to photographs in the superior temporal sulcus of the rhesus monkey

Neuronal correlates of motion-defined shape perception in primate dorsal and ventral streams

Human and non-human primates can readily perceive the shape of objects using visual motion. Classically, shape, and motion are considered to be separately processed via ventral and dorsal cortical pathways, respectively. However, many lines of anatomical and physiological evidence have indicated that these two pathways are likely to be interconnected at some stage. For motion-defined shape perception, these two pathways should interact with each other because the ventral pathway must utilize motion, which the dorsal pathway processes, to extract shape signal. However, it is unknown how interactions between cortical pathways are involved in neural mechanisms underlying motion-defined shape perception. We review evidence from psychophysical, lesion, neuroimaging and physiological research on motion-defined shape perception and then discuss the effects of behavioral demands on neural activity in ventral and dorsal cortical areas. Further, we discuss functions of two candidate sets of levels: early and higher-order cortical areas. The extrastriate area V4 and middle temporal (MT) area, which are reciprocally connected, at the early level are plausible areas for extracting the shape and/or constituent parts of shape from motion cues because neural dynamics are different from those during luminance-defined shape perception. On the other hand, among other higher-order visual areas, the anterior superior temporal sulcus likely contributes to the processing of cue-invariant shape recognition rather than cue-dependent shape processing. We suggest that sharing information about motion and shape between the early visual areas in the dorsal and ventral pathways is dependent on visual cues and behavioral requirements, indicating the interplay between the pathways.

Modulation of neuronal activity with cue-invariant shape discrimination in the primate superior temporal sulcus

Shape perception can be achieved based on various cues such as luminance, color, texture, depth and motion. To investigate common neural mechanisms underlying shape perception cued by various visual attributes, we examined single-neuron activity in the monkey anterior superior temporal sulcus (STS) in response to shapes defined by luminance and motion cues during shape discrimination. We found cortical mapping with respect to selectivity for shapes as well as for direction of motion in the STS. About 90% of shape-selective neurons were located in the lower bank of STS (lSTS) assigned to the ventral pathway, while about 80% of direction-selective neurons existed in the upper bank of STS (uSTS) assigned to the dorsal pathway. The neurons showing selectivity for both shape and motion coexisted in lSTS as well as uSTS. This result indicates that integration or convergence of shape information and motion information can occur in both banks of STS. About 90% of STS neurons showing selectivity both for shapes defined by luminance cue and for shapes defined by motion cue were located in lSTS. They showed a highly similar shape preference between the different visual attributes, indicating cue-invariant shape selectivity. The cue-invariant shape-selectivity was modulated with target selection as well as with discrimination performance of monkeys. These results suggest that lSTS could be involved in cue-invariant shape discrimination, but not the uSTS.

Is face recognition not so unique after all?

In monkeys, a number of different neocortical as well as limbic structures have cell populations that respond preferentially to face stimuli. Face selectivity is also differentiated within itself: Cells in the inferior temporal and prefrontal cortex tend to respond to facial identity, others in the upper bank of the superior temporal sulcus to gaze directions, and yet another population in the amygdala to facial expression. The great majority of these cells are sensitive to the entire configuration of a face. Changing the spatial arrangement of the facial features greatly diminishes the neurons' response. It would appear, then, that an entire neural network for faces exists which contains units highly selective to complex configurations and that respond to different aspects of the object "face." Given the vital importance of face recognition in primates, this may not come as a surprise. But are faces the only objects represented in this way? Behavioural work in humans suggests that nonface objects may be processed like faces if subjects are required to discriminate between visually similar exemplars and acquire sufficient expertise in doing so. Recent neuroimaging studies in humans indicate that level of categorisation and expertise interact to produce the specialisation for faces in the middle fusiform gyrus. Here we discuss some new evidence in the monkey suggesting that any arbitrary homogeneous class of artificial objects-which the animal has to individually learn, remember, and recognise again and again from among a large number of distractors sharing a number of common features with the target-can induce configurational selectivity in the response of neurons in the visual system. For all of the animals tested, the neurons from which we recorded were located in the anterior inferotemporal cortex. However, as we have only recorded from the posterior and anterior ventrolateral temporal lobe, other cells with a similar selectivity for the same objects may also exist in areas of the medial temporal lobe or in the limbic structures of the same "expert" monkeys. It seems that the encoding scheme used for faces may also be employed for other classes with similar properties. Thus, regarding their neural encoding, faces are not "special" but rather the "default special" class in the primate recognition system.

Invariant Visual Object and Face Recognition: Neural and Computational Bases, and a Model, VisNet

Neurophysiological evidence for invariant representations of objects and faces in the primate inferior temporal visual cortex is described. Then a computational approach to how invariant representations are formed in the brain is described that builds on the neurophysiology. A feature hierarchy model in which invariant representations can be built by self-organizing learning based on the temporal and spatial statistics of the visual input produced by objects as they transform in the world is described. VisNet can use temporal continuity in an associative synaptic learning rule with a short-term memory trace, and/or it can use spatial continuity in continuous spatial transformation learning which does not require a temporal trace. The model of visual processing in the ventral cortical stream can build representations of objects that are invariant with respect to translation, view, size, and also lighting. The model has been extended to provide an account of invariant representations in the dorsal visual system of the global motion produced by objects such as looming, rotation, and object-based movement. The model has been extended to incorporate top-down feedback connections to model the control of attention by biased competition in, for example, spatial and object search tasks. The approach has also been extended to account for how the visual system can select single objects in complex visual scenes, and how multiple objects can be represented in a scene. The approach has also been extended to provide, with an additional layer, for the development of representations of spatial scenes of the type found in the hippocampus.

The representation of information about faces in the temporal and frontal lobes

Neurophysiological evidence is described showing that some neurons in the macaque inferior temporal visual cortex have responses that are invariant with respect to the position, size and view of faces and objects, and that these neurons show rapid processing and rapid learning. Which face or object is present is encoded using a distributed representation in which each neuron conveys independent information in its firing rate, with little information evident in the relative time of firing of different neurons. This ensemble encoding has the advantages of maximising the information in the representation useful for discrimination between stimuli using a simple weighted sum of the neuronal firing by the receiving neurons, generalisation and graceful degradation. These invariant representations are ideally suited to provide the inputs to brain regions such as the orbitofrontal cortex and amygdala that learn the reinforcement associations of an individual's face, for then the learning, and the appropriate social and emotional responses, generalise to other views of the same face. A theory is described of how such invariant representations may be produced in a hierarchically organised set of visual cortical areas with convergent connectivity. The theory proposes that neurons in these visual areas use a modified Hebb synaptic modification rule with a short-term memory trace to capture whatever can be captured at each stage that is invariant about objects as the objects change in retinal view, position, size and rotation. Another population of neurons in the cortex in the superior temporal sulcus encodes other aspects of faces such as face expression, eye gaze, face view and whether the head is moving. These neurons thus provide important additional inputs to parts of the brain such as the orbitofrontal cortex and amygdala that are involved in social communication and emotional behaviour. Outputs of these systems reach the amygdala, in which face-selective neurons are found, and also the orbitofrontal cortex, in which some neurons are tuned to face identity and others to face expression. In humans, activation of the orbitofrontal cortex is found when a change of face expression acts as a social signal that behaviour should change; and damage to the orbitofrontal cortex can impair face and voice expression identification, and also the reversal of emotional behaviour that normally occurs when reinforcers are reversed.

Cerebral cortical control of orbicularis oculi motoneurons

Cerebral cortical neural networks associated with eyelid movement play a critical role in facial animation, contribute to the regulation of blink frequency, and help prevent ocular injury. Eyelid closure depends, in part, on motoneurons that innervate the orbicularis oculi (OO) muscles. In this study, OO motoneuron cortical afferents were identified in rhesus monkeys with rabies virus, a retrograde transneuronal tracer. Virus was injected into the right OO muscle and immunohistochemically localized after 4-6 day transport intervals. Labeled motoneurons were limited to dorsal portions of the ipsilateral facial motor nucleus. After 4- and 4.5-day transport intervals, most labeled cortical neurons were localized to ventrolateral premotor (LPMCv), dorsolateral premotor (LPMCd), and motor (M1) cortices. Labeled neurons were more sparsely distributed in supplementary (M2), caudal (M4), and rostral (M3) cingulate motor cortices; the frontal eye fields (FEF); pre-supplementary motor cortex (pre-SMA); somatosensory cortices (areas 3a, 3b, and 1); and prefrontal cortex. At longer transport intervals (5-6 days), labeled neurons increased substantially in LPMCv, LPMCd, M2, M3, M4, pre-SMA, and FEF. Concentrations of labeled neurons also appeared in cortices along the lateral fissure and intraparietal sulcus. Overall, the densest collection of labeled neurons was localized to the caudal junction of LPMCd and LPMCv with M1. Rostral M3 was another focus of OO premotor neurons. Labeled neurons were distributed bilaterally in all motor cortical areas with a modest contralateral predominance for M2, LPMC, and M1. Thus, the cortical control of OO motor activity is distributed bilaterally among multiple motor areas.

Effects of facial identity, facial expression, and subject's sex on laterality in monkeys

Previously we showed that rhesus monkeys processed discriminations of monkey faces significantly better with the right hemisphere of the brain than with the left. The overall effects of the type of discrimination, i.e. facial identity or expression, and the sex of the subject on laterality are examined here for seven phases of this series of experiments. Both types of discrimination produced a right hemispheric advantage, with slightly greater laterality for expression, but generally the laterality did not differ significantly for the two types. Female monkeys demonstrated more consistent and significant right hemispheric laterality than did males. Furthermore, female monkeys tended to be more lateralised for discriminations of expression, whereas males were about equally lateralised for both types. Thus, in these experiments the overall right hemispheric advantage for facial discriminations in monkeys reflects the contribution of the female subjects, especially when discriminating expression, more than that of the males.

The distinct modes of vision offered by feedforward and recurrent processing

An analysis of response latencies shows that when an image is presented to the visual system, neuronal activity is rapidly routed to a large number of visual areas. However, the activity of cortical neurons is not determined by this feedforward sweep alone. Horizontal connections within areas, and higher areas providing feedback, result in dynamic changes in tuning. The differences between feedforward and recurrent processing could prove pivotal in understanding the distinctions between attentive and pre-attentive vision as well as between conscious and unconscious vision. The feedforward sweep rapidly groups feature constellations that are hardwired in the visual brain, yet is probably incapable of yielding visual awareness; in many cases, recurrent processing is necessary before the features of an object are attentively grouped and the stimulus can enter consciousness.

During pain-avoidance neurons activated in the macaque anterior cingulate and caudate

Lesions in either the anterior cingulate cortex (ACC) or the caudate nucleus (CN) impair avoidance behavior from noxious somatic stimuli, so these two areas may play a similar role in pain-avoidance behavior. To test this hypothesis, we recorded single neuronal activities in the ACC and in the CN of a monkey while it was performing a pain-avoidance task. Ten of 136 ACC and eleven of 160 CN neurons responded selectively during pain-avoidance behavior. We found little difference in the population distribution or in the response latency and duration. Our present findings are in accordance with previous lesion and anatomical studies which suggest that these two regions could function as one module in pain-avoidance behavior.

Global and fine information coded by single neurons in the temporal visual cortex

When we see a person's face, we can easily recognize their species, individual identity and emotional state. How does the brain represent such complex information? A substantial number of neurons in the macaque temporal cortex respond to faces. However, the neuronal mechanisms underlying the processing of complex information are not yet clear. Here we recorded the activity of single neurons in the temporal cortex of macaque monkeys while presenting visual stimuli consisting of geometric shapes, and monkey and human faces with various expressions. Information theory was used to investigate how well the neuronal responses could categorize the stimuli. We found that single neurons conveyed two different scales of facial information in their firing patterns, starting at different latencies. Global information, categorizing stimuli as monkey faces, human faces or shapes, was conveyed in the earliest part of the responses. Fine information about identity or expression was conveyed later, beginning on average 51 ms after global information. We speculate that global information could be used as a 'header' to prepare destination areas for receiving more detailed information.

Categorization of complex visual images by rhesus monkeys. Part 2: single-cell study

In order to investigate the neural coding of ordinate-level visual categories, single-cell recordings were made in the anterior temporal cortex of two rhesus monkeys performing a categorization of colour images of trees versus images of other objects. Neurons showed a high average degree of selectivity for these complex colour images. Although most neurons responded to trees and non-trees, about a quarter responded in a category-specific manner, e.g. to trees but not non-trees, and about one-tenth responded almost exclusively to exemplars of the trained category. The responses of these neurons were largely invariant for stimulus transformations, e. g. changes in position or size, and decreased with the degree of image scrambling, mimicking the behavioural results. However, the responses of single neurons were insufficiently stimulus invariant to accommodate the entire range of variability present in the features of exemplars within the same category. This strong within-category selectivity challenges the idea that a prototype is represented at the single neuron level, but suggests that ordinate-level categorization is based on a population of neurons, each selective for a limited set of exemplars.

Neurons in the temporal cortex changed their preferred direction of motion dependent on shape

To investigate neuronal mechanisms that integrate different visual modalities such as motion and shape, neuronal activities in the superior temporal polysensory area (STP) were recorded from monkeys that were watching rotating images. In total, 194 neurons were identified as visually responsive. Of these, 73 neurons (38%) showed differential response depending on both shape and direction of motion (MS neurons). Of these 73 neurons, 21 (29%) were identified as reversal type MS neurons (MSr neurons), that is, they responded to an opposite preferred direction when the shape was different. The results confirm that neurons in the STP can be simultaneously activated by different attributes of visual stimuli. The data also suggest that individual STP neurons can process more than one type of visual stimulus.

ERPs and chronometry of face recognition: following-up Seeck et al. and George et al

Seeck et al. found that event-related potentials (ERPs) evoked by repeated and non-repeated face photographs differ as early as 50-70ms post-onset. They thus suggested that faces are recognized at these latencies, in contrast with current opinions in ERP literature. However, the similar latencies obtained by George et al. for stimuli not perceived as faces suggest that Seeck et al.'s differences could index repetition rather than face recognition per se. To address this issue, we used matched faces of known and unknown persons. We found the earliest differences between the ERPs to these faces between 76 and 130 ms. These results, which are consistent with other data, suggest that the differentiation of faces takes approximately 100 ms of processing time in humans.

Nociceptive neurons in the macaque anterior cingulate activate during anticipation of pain

Since the anterior cingulate cortex (ACC) is known to be involved both in nociception and in anticipation that precedes the avoidance of aversive stimuli, the linking of these functions may be processed in the ACC. To test this hypothesis, we recorded single neuronal activities in the ACC of a macaque monkey while it was performing a pain-avoidance task and examined them with nociceptive cutaneous electric stimuli (ES). Thirty-six neurons responded in anticipation of the ES. Of these, 22 neurons were tested with the ES and 11 responded. These neurons could be those that are involved both in nociception and in pain anticipation that precedes the avoidance of noxious stimuli.

Signal timing across the macaque visual system

The onset latencies of single-unit responses evoked by flashing visual stimuli were measured in the parvocellular (P) and magnocellular (M) layers of the dorsal lateral geniculate nucleus (LGNd) and in cortical visual areas V1, V2, V3, V4, middle temporal area (MT), medial superior temporal area (MST), and in the frontal eye field (FEF) in individual anesthetized monkeys. Identical procedures were carried out to assess latencies in each area, often in the same monkey, thereby permitting direct comparisons of timing across areas. This study presents the visual flash-evoked latencies for cells in areas where such data are common (V1 and V2), and are therefore a good standard, and also in areas where such data are sparse (LGNd M and P layers, MT, V4) or entirely lacking (V3, MST, and FEF in anesthetized preparation). Visual-evoked onset latencies were, on average, 17 ms shorter in the LGNd M layers than in the LGNd P layers. Visual responses occurred in V1 before any other cortical area. The next wave of activation occurred concurrently in areas V3, MT, MST, and FEF. Visual response latencies in areas V2 and V4 were progressively later and more broadly distributed. These differences in the time course of activation across the dorsal and ventral streams provide important temporal constraints on theories of visual processing.

A Cortical-type Modular Neural Network for Hypothetical Reasoning

We propose a multilayer neural network architecture that can implement the kind of hypothetical reasoning that the cortex seems to perform in making sense of the sensory input. The elementary processing nodes of each homogeneous sheet are not single formal neurons, but complex modules abducted from the functional organization of neocortical columns. As an example, we simulate face recognition in this neocortical architecture. A holistic but coarse initial hypothesis is generated by express forward input description and subsequently refined under the constraints of this hypothesis. Separation of forward input description and feedback generated hypothesis, while using the difference in both descriptions at each of the modular units to control the refinement, enables robust recognition and has the potential for autonomous learning. Copyright 1997 Elsevier Science Ltd.

The primate temporal pole: its putative role in object recognition and memory

In this article, we consider both the ventral temporopolar cortex and the perirhinal cortex (areas 35 and 36) as the anterior ventromedial temporal (aVMT) cortex, and discuss its role based on recent data in monkeys and human subjects. In monkeys, the aVMT cortex receives its primary input from area TE, and only minor input from other cortical areas. Laminar patterns of connections suggest that the aVMT cortex is a hierarchically higher-order area than area TE. Lesions of this cortex produce deficits in the learning and performance of visual memory tasks. Neurons in the aVMT cortex respond selectively to complex stimuli and changes in activity related to visual memory tasks. In humans, damage of this cortex induces deficits in the recognition of familiar objects and faces. The aVMT cortex is activated during recognition of familiar faces. In addition, the aVMT cortex is one of the most vulnerable areas in Alzheimer's disease. All these data indicate that the aVMT cortex is a higher-order visual cortical area that is related to object recognition and memory. The anterior area TE has been implicated in both functions. We propose here that these areas and the anterior entorhinal cortex are designated as the temporal pole, a brain region which is specialized for both object recognition and memory.

Visual neurons with higher selectivity can retain memory in the monkey temporal cortex

Activities of individual neurons were recorded from the superior temporal sulcus (STS) of rhesus monkeys during the performance of a visual discrimination and memory task. Of 174 neurons analyzed in detail, 19 neurons showed sustained changes in discharge rates during the delay period (D neurons). All the D neurons showed responses during the presentation of the same stimulus and had higher selectivity compared to the remaining non-D neurons. The data indicated that a subgroup of highly selective visual neurons in the STS participate in short-term retention of these stimuli.