The monkey ventral premotor cortex processes 3D shape from disparity

Large-scale cortico-cerebellar computations for horizontal and vertical vergence in humans

Horizontal and vertical vergence eye movements play a central role in binocular coordination. Neurophysiological studies suggest that cortical and subcortical regions in animals and humans are involved in horizontal vergence. However, little is known about the extent to which the neural mechanism underlying vertical vergence overlaps with that of horizontal vergence. In this study, to explore neural computation for horizontal and vertical vergence, we simultaneously recorded electrooculography (EOG) and whole-head magnetoencephalography (MEG) while presenting large-field stereograms for 29 healthy human adults. The stereograms were designed to produce vergence responses by manipulating horizontal and vertical binocular disparities. A model-based approach was used to assess neural sensitivity to horizontal and vertical disparities via MEG source estimation and the theta-band (4 Hz) coherence between brain activity and EOG vergence velocity. We found similar time-locked neural responses to horizontal and vertical disparity in cortical and cerebellar areas at around 100–250 ms after stimulus onset. In contrast, the low-frequency oscillatory neural activity associated with the execution of vertical vergence differed from that of horizontal vergence. These findings indicate that horizontal and vertical vergence involve partially shared but distinct computations in large-scale cortico-cerebellar networks.

The Causal Role of Three Frontal Cortical Areas in Grasping

Efficient object grasping requires the continuous control of arm and hand movements based on visual information. Previous studies have identified a network of parietal and frontal areas that is crucial for the visual control of prehension movements. Electrical microstimulation of 3D shape-selective clusters in AIP during functional magnetic resonance imaging activates areas F5a and 45B, suggesting that these frontal areas may represent important downstream areas for object processing during grasping, but the role of area F5a and 45B in grasping is unknown. To assess their causal role in the frontal grasping network, we reversibly inactivated 45B, F5a, and F5p during visually guided grasping in macaque monkeys. First, we recorded single neuron activity in 45B, F5a, and F5p to identify sites with object responses during grasping. Then, we injected muscimol or saline to measure the grasping deficit induced by the temporary disruption of each of these three nodes in the grasping network. The inactivation of all three areas resulted in a significant increase in the grasping time in both animals, with the strongest effect observed in area F5p. These results not only confirm a clear involvement of F5p, but also indicate causal contributions of area F5a and 45B in visually guided object grasping.

Unique Neural Activity Patterns Among Lower Order Cortices and Shared Patterns Among Higher Order Cortices During Processing of Similar Shapes With Different Stimulus Types

We investigated the neural mechanism of the processing of three-dimensional (3D) shapes defined by disparity and perspective. We measured blood oxygenation level-dependent signals as participants viewed and classified 3D images of convex-concave shapes. According to the cue (disparity or perspective) and element type (random dots or black and white dotted lines), three types of stimuli were used: random dot stereogram, black and white dotted lines with perspective, and black and white dotted lines with binocular disparity. The blood oxygenation level-dependent images were then classified by multivoxel pattern analysis. To identify areas selective to shape, we assessed convex-concave classification accuracy with classifiers trained and tested using signals evoked by the same stimulus type (same cue and element type). To identify cortical regions with similar neural activity patterns regardless of stimulus type, we assessed the convex-concave classification accuracy of transfer classification in which classifiers were trained and tested using different stimulus types (different cues or element types). Classification accuracy using the same stimulus type was high in the early visual areas and subregions of the intraparietal sulcus (IPS), whereas transfer classification accuracy was high in the dorsal subregions of the IPS. These results indicate that the early visual areas process the specific features of stimuli, whereas the IPS regions perform more generalized processing of 3D shapes, independent of a specific stimulus type.

Intrinsic functional clustering of ventral premotor F5 in the macaque brain

Neurophysiological and anatomical data suggest the existence of several functionally distinct regions in the lower arcuate sulcus and adjacent postarcuate convexity of the macaque monkey. Ventral premotor F5c lies on the postarcuate convexity and consists of a dorsal hand-related and ventral mouth-related field. The posterior bank of the lower arcuate contains two additional premotor F5 subfields at different anterior-posterior levels, F5a and F5p. Anterior to F5a, area 44 has been described as a dysgranular zone occupying the deepest part of the fundus of the inferior arcuate. Finally, area GrFO occupies the most rostral portion of the fundus and posterior bank of inferior arcuate and extends ventrally onto the frontal operculum. Recently, data-driven exploratory approaches using resting-state fMRI data have been suggested as a promising non-invasive method for examining the functional organization of the primate brain. Here, we examined to what extent partitioning schemes derived from data-driven clustering analysis of resting-state fMRI data correspond with the proposed organization of the fundus and posterior bank of the macaque arcuate sulcus, as suggested by invasive architectonical, connectional and functional investigations. Using a hierarchical clustering analysis, we could retrieve clusters corresponding to the dorsal and ventral portions of F5c on the postarcuate convexity, F5a and F5p at different antero-posterior locations on the posterior bank of the lower arcuate, area 44 in the fundus, as well as part of area GrFO in the most anterior portion of the fundus. Additionally, each of these clusters displayed distinct whole-brain functional connectivity, in line with previous anatomical tracer and seed-based functional connectivity investigations of F5/44 subdivisions. Overall, our data suggests that hierarchical clustering analysis of resting-state fMRI data can retrieve a fine-grained level of cortical organization that resembles detailed parcellation schemes derived from invasive functional and anatomical investigations.

Effective Connectivity Reveals an Interconnected Inferotemporal Network for Three-Dimensional Structure Processing

Disparity-defined 3D shape is processed in both the ventral and the dorsal visual stream. The network of cortical areas that is activated during the processing of disparity-defined 3D shape includes, in addition to parietal and premotor areas, three clearly distinct regions in inferotemporal cortex (ITC). To investigate the connectivity of the latter regions, we combined electrical stimulation with fMRI in male macaque monkeys. Electrical stimulation of each of the 3D-structure nodes in ITC mainly elicited increased fMRI activations in the other 3D-structure nodes and more variably in other parts of ventral visual cortex. Importantly, no increased activation was found in parietal areas, nor in PFC, whereas microstimulation in posterior parietal cortex did activate the ITC. Our results indicate that 3D-structure nodes in ITC form a strongly interconnected network, receiving input from parietal areas implicated in 3D-structure processing.SIGNIFICANCE STATEMENT Previous studies combining electrical microstimulation with functional imaging showed an interconnected set of regions in the ventral stream processing faces or bodies, but is has been unclear whether the same is true for other visual categories. Here the authors show that there is a connected system of stereo-selective regions in inferotemporal cortex, receiving input from parietal areas in the dorsal stream.

Vertical size disparity induces enhanced neural responses in good stereo observers

Stereoscopic three-dimensional vision requires cortical processing for horizontal binocular disparity between the two eyes' retinal images. Behavioral and theoretical studies suggest that vertical size disparity is used to recover the viewing geometry and to generate the slant of a large surface. However, unlike horizontal disparity, the relation between stereopsis and neural responses to vertical disparity remains controversial. To determine the role of cortical processing for vertical size disparity in stereopsis, we measured neuromagnetic responses to disparities in people with good and poor stereopsis, using magnetoencephalography (MEG). Healthy adult participants viewed stereograms with a horizontal or vertical size disparity, and judged the perceived slant of the pattern. We assessed neural activity in response to disparities in the visual cortex and the phase locking of oscillatory responses including the alpha frequency range using MEG. For participants with good stereopsis, activity in the visual areas was significantly higher in response to vertical size disparity than to horizontal size disparity. The time-frequency analysis revealed that early neural responses to vertical size disparity were more phase-locked in good stereo participants than in poor stereo participants. These results provide neuromagnetic evidence that vertical-size disparity processing plays a role in good stereo vision.

Single-cell selectivity and functional architecture of human lateral occipital complex

The human lateral occipital complex (LOC) is more strongly activated by images of objects compared to scrambled controls, but detailed information at the neuronal level is currently lacking. We recorded with microelectrode arrays in the LOC of 2 patients and obtained highly selective single-unit, multi-unit, and high-gamma responses to images of objects. Contrary to predictions derived from functional imaging studies, all neuronal properties indicated that the posterior subsector of LOC we recorded from occupies an unexpectedly high position in the hierarchy of visual areas. Notably, the response latencies of LOC neurons were long, the shape selectivity was spatially clustered, LOC receptive fields (RFs) were large and bilateral, and a number of LOC neurons exhibited three-dimensional (3D)-structure selectivity (a preference for convex or concave stimuli), which are all properties typical of end-stage ventral stream areas. Thus, our results challenge prevailing ideas about the position of the more posterior subsector of LOC in the hierarchy of visual areas.

Identifying Cortical Substrates Underlying the Phenomenology of Stereopsis and Realness: A Pilot fMRI Study

Viewing a real scene or a stereoscopic image (e.g., 3D movies) with both eyes yields a vivid subjective impression of object solidity, tangibility, immersive negative space and sense of realness; something that is not experienced when viewing single pictures of 3D scenes normally with both eyes. This phenomenology, sometimes referred to as stereopsis, is conventionally ascribed to the derivation of depth from the differences in the two eye's images (binocular disparity). Here we report on a pilot study designed to explore if dissociable neural activity associated with the phenomenology of realness can be localized in the cortex. In order to dissociate subjective impression from disparity processing, we capitalized on the finding that the impression of realness associated with stereoscopic viewing can also be generated when viewing a single picture of a 3D scene with one eye through an aperture. Under a blocked fMRI design, subjects viewed intact and scrambled images of natural 3-D objects, and scenes under three viewing conditions: (1) single pictures viewed normally with both eyes (binocular); (2) single pictures viewed with one eye through an aperture (monocular-aperture); and (3) stereoscopic anaglyph images of the same scenes viewed with both eyes (binocular stereopsis). Fixed-effects GLM contrasts aimed at isolating the phenomenology of stereopsis demonstrated a selective recruitment of similar posterior parietal regions for both monocular and binocular stereopsis conditions. Our findings provide preliminary evidence that the cortical processing underlying the subjective impression of realness may be dissociable and distinct from the derivation of depth from disparity.

Dissociating neural activity associated with the subjective phenomenology of monocular stereopsis: An EEG study

The subjective phenomenology associated with stereopsis, of solid tangible objects separated by a palpable negative space, is conventionally thought to be a by-product of the derivation of depth from binocular disparity. However, the same qualitative impression has been reported in the absence of disparity, e.g., when viewing pictorial images monocularly through an aperture. Here we aimed to explore if we could identify dissociable neural activity associated with the qualitative impression of stereopsis in the absence of the processing of binocular disparities. We measured EEG activity while subjects viewed pictorial (non-stereoscopic) images of 2D and 3D geometric forms under four different viewing conditions (binocular, monocular, binocular aperture, monocular aperture). EEG activity was analysed by oscillatory source localization (beamformer technique) to examine power change in occipital and parietal regions across viewing and stimulus conditions in targeted frequency bands (alpha: 8-13 Hz & gamma: 60-90 Hz). We observed expected event-related gamma synchronization and alpha desynchronization in occipital cortex and predominant gamma synchronization in parietal cortex across viewing and stimulus conditions. However, only the viewing condition predicted to generate the strongest impression of stereopsis (monocular aperture) revealed significantly elevated gamma synchronization within the parietal cortex for the critical contrasts (3D vs. 2D form). These findings suggest dissociable neural processes specific to the qualitative impression of stereopsis as distinguished from disparity processing.

Functional specialization of macaque premotor F5 subfields with respect to hand and mouth movements: A comparison of task and resting-state fMRI

Based on architectonic, tract-tracing or functional criteria, the rostral portion of ventral premotor cortex in the macaque monkey, also termed area F5, has been divided into several subfields. Cytoarchitectonical investigations suggest the existence of three subfields, F5c (convexity), F5p (posterior) and F5a (anterior). Electrophysiological investigations have suggested a gradual dorso-ventral transition from hand- to mouth-dominated motor fields, with F5p and ventral F5c strictly related to hand movements and mouth movements, respectively. The involvement of F5a in this respect, however, has received much less attention. Recently, data-driven resting-state fMRI approaches have also been used to examine the presence of distinct functional fields in macaque ventral premotor cortex. Although these studies have suggested several functional clusters in/near macaque F5, so far the parcellation schemes derived from these clustering methods do not completely retrieve the same level of F5 specialization as suggested by aforementioned invasive techniques. Here, using seed-based resting-state fMRI analyses, we examined the functional connectivity of different F5 seeds with key regions of the hand and face/mouth parieto-frontal-insular motor networks. In addition, we trained monkeys to perform either hand grasping or ingestive mouth movements in the scanner in order to compare resting-state with task-derived functional hand and mouth motor networks. In line with previous single-cell investigations, task-fMRI suggests involvement of F5p, dorsal F5c and F5a in the execution of hand grasping movements, while non-communicative mouth movements yielded particularly pronounced responses in ventral F5c. Corroborating with anatomical tracing data of macaque F5 subfields, seed-based resting-state fMRI suggests a transition from predominant functional correlations with the hand-motor network in F5p to mostly mouth-motor network functional correlations in ventral F5c. Dorsal F5c yielded robust functional correlations with both hand- and mouth-motor networks. In addition, the deepest part of the fundus of the inferior arcuate, corresponding to area 44, displayed a strikingly different functional connectivity profile compared to neighboring F5a, suggesting a different functional specialization for these two neighboring regions.

Towards a unified perspective of object shape and motion processing in human dorsal cortex

Although object-related areas were discovered in human parietal cortex a decade ago, surprisingly little is known about the nature and purpose of these representations, and how they differ from those in the ventral processing stream. In this article, we review evidence for the unique contribution of object areas of dorsal cortex to three-dimensional (3-D) shape representation, the localization of objects in space, and in guiding reaching and grasping actions. We also highlight the role of dorsal cortex in form-motion interaction and spatiotemporal integration, possible functional relationships between 3-D shape and motion processing, and how these processes operate together in the service of supporting goal-directed actions with objects. Fundamental differences between the nature of object representations in the dorsal versus ventral processing streams are considered, with an emphasis on how and why dorsal cortex supports veridical (rather than invariant) representations of objects to guide goal-directed hand actions in dynamic visual environments.

Hierarchical Organization Within the Ventral Premotor Cortex of the Macaque Monkey

Recent studies have revealed that the ventral premotor cortex (PMv) of nonhuman primates plays a pivotal role in various behaviors that require the transformation of sensory cues to appropriate actions. Examples include decision-making based on various sensory cues, preparation for upcoming motor behavior, adaptive sensorimotor transformation, and the generation of motor commands using rapid sensory feedback. Although the PMv has frequently been regarded as a single entity, it can be divided into at least five functionally distinct regions: F4, a dorsal convexity region immediately rostral to the primary motor cortex (M1); F5p, a cortical region immediately rostral to F4, lying within the arcuate sulcus; F5c, a ventral convexity region rostral to F4; and F5a, located in the caudal bank of the arcuate sulcus inferior limb lateral to F5p. Among these, F4 can be further divided into dorsal and ventral subregions (F4d and F4v), which are involved in forelimb and orofacial movements, respectively. F5p contains "mirror neurons" to understand others' actions based on visual and other types of information, and F4d and F5p work together as a functional complex involved in controlling forelimb and eye movements, most efficiently in the execution and completion of coordinated eye-hand movements for reaching and grasping under visual guidance. In contrast, F5c and F5a are hierarchically higher than the F4d, F5p, and F5v complexes, and play a role in decision-making based on various sensory discriminations. Hence, the PMv subregions form a hierarchically organized integral system from decision-making to eye-hand coordination under various behavioral circumstances.

Thresholds for sine-wave corrugations defined by binocular disparity in random dot stereograms: Factor analysis of individual differences reveals two stereoscopic mechanisms tuned for spatial frequency

Threshold functions for sinusoidal depth corrugations typically reach their minimum (highest sensitivity) at spatial frequencies of 0.2-0.4 cycles/degree (cpd), with lower thresholds for horizontal than vertical corrugations at low spatial frequencies. To elucidate spatial frequency and orientation tuning of stereoscopic mechanisms, we measured the disparity sensitivity functions, and used factor analytic techniques to estimate the existence of independent underlying stereo channels. The data set (N = 30 individuals) was for horizontal and vertical corrugations of spatial frequencies ranging from 0.1 to 1.6 cpd. A principal component analysis of disparity sensitivities (log-arcsec) revealed that two significant factors accounted for 70% of the variability. Following Varimax rotation to approximate "simple structure", one factor clearly loaded onto low spatial frequencies (≤0.4 cpd), and a second was tuned to higher spatial frequencies (≥0.8 cpd). Each factor had nearly identical tuning (loadings) for horizontal and vertical patterns. The finding of separate factors for low and high spatial frequencies is consistent with previous studies. The failure to find separate factors for horizontal and vertical corrugations is somewhat surprising because the neuronal mechanisms are believed to be different. Following an oblique rotation (Direct Oblimin), the two factors correlated significantly, suggesting some interdependence rather than full independence between the two factors.

Binocular depth processing in the ventral visual pathway

One of the most powerful forms of depth perception capitalizes on the small relative displacements, or binocular disparities, in the images projected onto each eye. The brain employs these disparities to facilitate various computations, including sensori-motor transformations (reaching, grasping), scene segmentation and object recognition. In accordance with these different functions, disparity activates a large number of regions in the brain of both humans and monkeys. Here, we review how disparity processing evolves along different regions of the ventral visual pathway of macaques, emphasizing research based on both correlational and causal techniques. We will discuss the progression in the ventral pathway from a basic absolute disparity representation to a more complex three-dimensional shape code. We will show that, in the course of this evolution, the underlying neuronal activity becomes progressively more bound to the global perceptual experience. We argue that these observations most probably extend beyond disparity processing per se, and pertain to object processing in the ventral pathway in general. We conclude by posing some important unresolved questions whose answers may significantly advance the field, and broaden its scope.This article is part of the themed issue 'Vision in our three-dimensional world'.

Functional interactions between the macaque dorsal and ventral visual pathways during three-dimensional object vision

The division of labor between the dorsal and the ventral visual stream in the primate brain has inspired numerous studies on the visual system in humans and in nonhuman primates. However, how and under which circumstances the two visual streams interact is still poorly understood. Here we review evidence from anatomy, modelling, electrophysiology, electrical microstimulation (EM), reversible inactivation and functional imaging in the macaque monkey aimed at clarifying at which levels in the hierarchy of visual areas the two streams interact, and what type of information might be exchanged between the two streams during three-dimensional (3D) object viewing. Neurons in both streams encode 3D structure from binocular disparity, synchronized activity between parietal and inferotemporal areas is present during 3D structure categorization, and clusters of 3D structure-selective neurons in parietal cortex are anatomically connected to ventral stream areas. In addition, caudal intraparietal cortex exerts a causal influence on 3D-structure related activations in more anterior parietal cortex and in inferotemporal cortex. Thus, both anatomical and functional evidence indicates that the dorsal and the ventral visual stream interact during 3D object viewing.

A brief comparative review of primate posterior parietal cortex: A novel hypothesis on the human toolmaker

The primate visual system contains two major cortical pathways: a ventral-temporal pathway that has been associated with object processing and recognition, and a dorsal-parietal pathway that has been associated with spatial processing and action guidance. Our understanding of the role of the dorsal pathway, in particular, has greatly evolved within the framework of the two-pathway hypothesis since its original conception. Here, we present a comparative review of the primate dorsal pathway in humans and monkeys based on electrophysiological, neuroimaging, neuropsychological, and neuroanatomical studies. We consider similarities and differences across species in terms of the topographic representation of visual space; specificity for eye, reaching, or grasping movements; multi-modal response properties; and the representation of objects and tools. We also review the relative anatomical location of functionally- and topographically-defined regions of the posterior parietal cortex. An emerging theme from this comparative analysis is that non-spatial information is represented to a greater degree, and with increased complexity, in the human dorsal visual system. We propose that non-spatial information in the primate parietal cortex contributes to the perception-to-action system aimed at manipulating objects in peripersonal space. In humans, this network has expanded in multiple ways, including the development of a dorsal object vision system mirroring the complexity of the ventral stream, the integration of object information with parietal working memory systems, and the emergence of tool-specific object representations in the anterior intraparietal sulcus and regions of the inferior parietal lobe. We propose that these evolutionary changes have enabled the emergence of human-specific behaviors, such as the sophisticated use of tools.

Brain activation difference evoked by different binocular disparities of stereograms: An fMRI study

The binocular disparity of two retina images is a main cue of stereoscopic vision. However, the global dependency between brain response and binocular disparity still remains unclear. Here, we used functional Magnetic Resonance Imaging (fMRI) to identify stereopsis-related brain regions with a modified Random Dot Stereogram (RDS) and plotted the activation variation curves under different disparity size. In order to eliminate the confounding shape difference between the stereogram and the plane, commonly seen in RDS, we modified the RDS to a checkerboard version. We found that V3A, V7 and MT+/V5 in dorsal visual stream were activated in stereoscopic experiment, while little activation was found in ventral visual regions. According to the activation trends, 13 subjects were divided into three groups: 5 subjects with turning points (a shift from increased to decreased activation), 5 subjects without turning points and 3 subjects with activation unrelated to disparity. We inferred that the dorsal visual stream primarily processes spatial depth information, rather than shape information.

Stereopsis after anterior temporal lobectomy

Brain areas critical for stereopsis have been investigated in non-human primates but are largely unknown in the human brain. Microelectrode recordings and functional MRI (fMRI) studies in monkeys have shown that in monkeys the inferior temporal cortex is critically involved in 3D shape categorization. Furthermore, some human fMRI studies similarly suggest an involvement of visual areas in the temporal lobe in depth perception. We aimed to investigate the role of the human anterior temporal neocortex in stereopsis by assessing stereoscopic depth perception before and after anterior temporal lobectomy. Eighteen epilepsy surgery patients were tested, pre- and postoperatively, in 3 different depth discrimination tasks. Sensitivity for local and global disparity was tested in a near-far discrimination task and sensitivity for 3D curvature was assessed in a convex-concave discrimination task, where 3D shapes were presented at different positions in depth. We found no evidence that temporal lobe epilepsy surgery has a significant effect on stereopsis. In contrast with earlier findings, we conclude that local as well as global stereopsis is maintained after unilateral resection of the temporal pole in epilepsy surgery patients. Our findings, together with previous studies, suggest that in humans more posterior visual regions underlie depth perception.

Posterior Parietal Cortex Drives Inferotemporal Activations During Three-Dimensional Object Vision

The primate visual system consists of a ventral stream, specialized for object recognition, and a dorsal visual stream, which is crucial for spatial vision and actions. However, little is known about the interactions and information flow between these two streams. We investigated these interactions within the network processing three-dimensional (3D) object information, comprising both the dorsal and ventral stream. Reversible inactivation of the macaque caudal intraparietal area (CIP) during functional magnetic resonance imaging (fMRI) reduced fMRI activations in posterior parietal cortex in the dorsal stream and, surprisingly, also in the inferotemporal cortex (ITC) in the ventral visual stream. Moreover, CIP inactivation caused a perceptual deficit in a depth-structure categorization task. CIP-microstimulation during fMRI further suggests that CIP projects via posterior parietal areas to the ITC in the ventral stream. To our knowledge, these results provide the first causal evidence for the flow of visual 3D information from the dorsal stream to the ventral stream, and identify CIP as a key area for depth-structure processing. Thus, combining reversible inactivation and electrical microstimulation during fMRI provides a detailed view of the functional interactions between the two visual processing streams.

3D Shape Perception in Posterior Cortical Atrophy: A Visual Neuroscience Perspective

Posterior cortical atrophy (PCA) is a rare focal neurodegenerative syndrome characterized by progressive visuoperceptual and visuospatial deficits, most often due to atypical Alzheimer's disease (AD). We applied insights from basic visual neuroscience to analyze 3D shape perception in humans affected by PCA. Thirteen PCA patients and 30 matched healthy controls participated, together with two patient control groups with diffuse Lewy body dementia (DLBD) and an amnestic-dominant phenotype of AD, respectively. The hierarchical study design consisted of 3D shape processing for 4 cues (shading, motion, texture, and binocular disparity) with corresponding 2D and elementary feature extraction control conditions. PCA and DLBD exhibited severe 3D shape-processing deficits and AD to a lesser degree. In PCA, deficient 3D shape-from-shading was associated with volume loss in the right posterior inferior temporal cortex. This region coincided with a region of functional activation during 3D shape-from-shading in healthy controls. In PCA patients who performed the same fMRI paradigm, response amplitude during 3D shape-from-shading was reduced in this region. Gray matter volume in this region also correlated with 3D shape-from-shading in AD. 3D shape-from-disparity in PCA was associated with volume loss slightly more anteriorly in posterior inferior temporal cortex as well as in ventral premotor cortex. The findings in right posterior inferior temporal cortex and right premotor cortex are consistent with neurophysiologically based models of the functional anatomy of 3D shape processing. However, in DLBD, 3D shape deficits rely on mechanisms distinct from inferior temporal structural integrity.

SIGNIFICANCE STATEMENT

Posterior cortical atrophy (PCA) is a neurodegenerative syndrome characterized by progressive visuoperceptual dysfunction and most often an atypical presentation of Alzheimer's disease (AD) affecting the ventral and dorsal visual streams rather than the medial temporal system. We applied insights from fundamental visual neuroscience to analyze 3D shape perception in PCA. 3D shape-processing deficits were affected beyond what could be accounted for by lower-order processing deficits. For shading and disparity, this was related to volume loss in regions previously implicated in 3D shape processing in the intact human and nonhuman primate brain. Typical amnestic-dominant AD patients also exhibited 3D shape deficits. Advanced visual neuroscience provides insight into the pathogenesis of PCA that also bears relevance for vision in typical AD.

Effects of Microstimulation in the Anterior Intraparietal Area during Three-Dimensional Shape Categorization

The anterior intraparietal area (AIP) of rhesus monkeys is part of the dorsal visual stream and contains neurons whose visual response properties are commensurate with a role in three-dimensional (3D) shape perception. Neuronal responses in AIP signal the depth structure of disparity-defined 3D shapes, reflect the choices of monkeys while they categorize 3D shapes, and mirror the behavioral variability across different stimulus conditions during 3D-shape categorization. However, direct evidence for a role of AIP in 3D-shape perception has been lacking. We trained rhesus monkeys to categorize disparity-defined 3D shapes and examined AIP's contribution to 3D-shape categorization by microstimulating in clusters of 3D-shape selective AIP neurons during task performance. We find that microstimulation effects on choices (monkey M1) and reaction times (monkey M1 and M2) depend on the 3D-shape preference of the stimulated site. Moreover, electrical stimulation of the same cells, during either the 3D-shape-categorization task or a saccade task, could affect behavior differently. Interestingly, in one monkey we observed a strong correlation between the strength of choice-related AIP activity (choice probabilities) and the influence of microstimulation on 3D-shape-categorization behavior (choices and reaction time). These findings propose AIP as part of the network responsible for 3D-shape perception. The results also show that the anterior intraparietal cortex contains cells with different tuning properties, i.e. 3D-shape- or saccade-related, that can be dynamically read out depending on the requirements of the task at hand.

Functional architecture for disparity in macaque inferior temporal cortex and its relationship to the architecture for faces, color, scenes, and visual field

Binocular disparity is a powerful depth cue for object perception. The computations for object vision culminate in inferior temporal cortex (IT), but the functional organization for disparity in IT is unknown. Here we addressed this question by measuring fMRI responses in alert monkeys to stimuli that appeared in front of (near), behind (far), or at the fixation plane. We discovered three regions that showed preferential responses for near and far stimuli, relative to zero-disparity stimuli at the fixation plane. These "near/far" disparity-biased regions were located within dorsal IT, as predicted by microelectrode studies, and on the posterior inferotemporal gyrus. In a second analysis, we instead compared responses to near stimuli with responses to far stimuli and discovered a separate network of "near" disparity-biased regions that extended along the crest of the superior temporal sulcus. We also measured in the same animals fMRI responses to faces, scenes, color, and checkerboard annuli at different visual field eccentricities. Disparity-biased regions defined in either analysis did not show a color bias, suggesting that disparity and color contribute to different computations within IT. Scene-biased regions responded preferentially to near and far stimuli (compared with stimuli without disparity) and had a peripheral visual field bias, whereas face patches had a marked near bias and a central visual field bias. These results support the idea that IT is organized by a coarse eccentricity map, and show that disparity likely contributes to computations associated with both central (face processing) and peripheral (scene processing) visual field biases, but likely does not contribute much to computations within IT that are implicated in processing color.

Shape representations in the primate dorsal visual stream

The primate visual system extracts object shape information for object recognition in the ventral visual stream. Recent research has demonstrated that object shape is also processed in the dorsal visual stream, which is specialized for spatial vision and the planning of actions. A number of studies have investigated the coding of 2D shape in the anterior intraparietal area (AIP), one of the end-stage areas of the dorsal stream which has been implicated in the extraction of affordances for the purpose of grasping. These findings challenge the current understanding of area AIP as a critical stage in the dorsal stream for the extraction of object affordances. The representation of three-dimensional (3D) shape has been studied in two interconnected areas known to be critical for object grasping: area AIP and area F5a in the ventral premotor cortex (PMv), to which AIP projects. In both areas neurons respond selectively to 3D shape defined by binocular disparity, but the latency of the neural selectivity is approximately 10 ms longer in F5a compared to AIP, consistent with its higher position in the hierarchy of cortical areas. Furthermore, F5a neurons were more sensitive to small amplitudes of 3D curvature and could detect subtle differences in 3D structure more reliably than AIP neurons. In both areas, 3D-shape selective neurons were co-localized with neurons showing motor-related activity during object grasping in the dark, indicating a close convergence of visual and motor information on the same clusters of neurons.

A human homologue of monkey F5c

Area F5c is a monkey premotor area housing mirror neurons which responds more strongly to grasping observation when the actor is visible than when only the actor's hand is visible. Here we used this characteristic fMRI signature of F5c in seven imaging experiments – one in macaque monkeys and six in humans – to identify the human homologue of monkey F5c. By presenting the two grasping actions (actor, hand) and varying the low level visual characteristics, we localized a putative human homologue of area F5c (phF5c) in the inferior part of precentral sulcus, bilaterally. In contrast to monkey F5c, phF5c is asymmetric, with a right-sided bias, and is activated more strongly during the observation of the later stages of grasping when the hand is close to the object. The latter characteristic might be related to the emergence, in humans, of the capacity to precisely copy motor acts performed by others, and thus imitation.

•

We use parallel fMRI to identify the human homologue of macaque F5c.

•

In premotor cortex only F5c reacts more to observing grasping with the actor visible.

•

Two bilateral inferior precentral sulcus sites respond similarly for many stimuli.

•

The human homologues of F5c are asymmetric and require fixation near the target.

Effective connectivity of depth-structure-selective patches in the lateral bank of the macaque intraparietal sulcus

Extrastriate cortical areas are frequently composed of subpopulations of neurons encoding specific features or stimuli, such as color, disparity, or faces, and patches of neurons encoding similar stimulus properties are typically embedded in interconnected networks, such as the attention or face-processing network. The goal of the current study was to examine the effective connectivity of subsectors of neurons in the same cortical area with highly similar neuronal response properties. We first recorded single- and multi-unit activity to identify two neuronal patches in the anterior part of the macaque intraparietal sulcus (IPS) showing the same depth structure selectivity and then employed electrical microstimulation during functional magnetic resonance imaging in these patches to determine the effective connectivity of these patches. The two IPS subsectors we identified-with the same neuronal response properties and in some cases separated by only 3 mm-were effectively connected to remarkably distinct cortical networks in both dorsal and ventral stream in three macaques. Conversely, the differences in effective connectivity could account for the known visual-to-motor gradient within the anterior IPS. These results clarify the role of the anterior IPS as a pivotal brain region where dorsal and ventral visual stream interact during object analysis. Thus, in addition to the anatomical connectivity of cortical areas and the properties of individual neurons in these areas, the effective connectivity provides novel key insights into the widespread functional networks that support behavior.

Choice-related Activity in the Anterior Intraparietal Area during 3-D Structure Categorization

The anterior intraparietal area (AIP) of macaques contains neurons that signal the depth structure of disparity-defined 3-D shapes. Previous studies have suggested that AIP's depth information is used for sensorimotor transformations related to the efficient grasping of 3-D objects. We trained monkeys to categorize disparity-defined 3-D shapes and examined whether neuronal activity in AIP may also underlie pure perceptual categorization behavior. We first show that neurons with a similar 3-D shape preference cluster in AIP. We then demonstrate that the monkeys' 3-D shape discrimination performance depends on the position in depth of the stimulus and that this performance difference is reflected in the activity of AIP neurons. We further reveal correlations between the neuronal activity in AIP and the subject's subsequent choices and RTs during 3-D shape categorization. Our findings propose AIP as an important processing stage for 3-D shape perception.

Neuroimaging of amblyopia and binocular vision: a review

Amblyopia is a cerebral visual impairment considered to derive from abnormal visual experience (e.g., strabismus, anisometropia). Amblyopia, first considered as a monocular disorder, is now often seen as a primarily binocular disorder resulting in more and more studies examining the binocular deficits in the patients. The neural mechanisms of amblyopia are not completely understood even though they have been investigated with electrophysiological recordings in animal models and more recently with neuroimaging techniques in humans. In this review, we summarize the current knowledge about the brain regions that underlie the visual deficits associated with amblyopia with a focus on binocular vision using functional magnetic resonance imaging. The first studies focused on abnormal responses in the primary and secondary visual areas whereas recent evidence shows that there are also deficits at higher levels of the visual pathways within the parieto-occipital and temporal cortices. These higher level areas are part of the cortical network involved in 3D vision from binocular cues. Therefore, reduced responses in these areas could be related to the impaired binocular vision in amblyopic patients. Promising new binocular treatments might at least partially correct the activation in these areas. Future neuroimaging experiments could help to characterize the brain response changes associated with these treatments and help devise them.

Merging functional and structural properties of the monkey auditory cortex

Recent neuroimaging studies in primates aim to define the functional properties of auditory cortical areas, especially areas beyond A1, in order to further our understanding of the auditory cortical organization. Precise mapping of functional magnetic resonance imaging (fMRI) results and interpretation of their localizations among all the small auditory subfields remains challenging. To facilitate this mapping, we combined here information from cortical folding, micro-anatomy, surface-based atlas and tonotopic mapping. We used for the first time, phase-encoded fMRI design for mapping the monkey tonotopic organization. From posterior to anterior, we found a high-low-high progression of frequency preference on the superior temporal plane. We show a faithful representation of the fMRI results on a locally flattened surface of the superior temporal plane. In a tentative scheme to delineate core versus belt regions which share similar tonotopic organizations we used the ratio of T1-weighted and T2-weighted MR images as a measure of cortical myelination. Our results, presented along a co-registered surface-based atlas, can be interpreted in terms of a current model of the monkey auditory cortex.

The transition in the ventral stream from feature to real-world entity representations

We propose that the ventral visual pathway of human and non-human primates is organized into three levels: (1) ventral retinotopic cortex including what is known as TEO in the monkey but corresponds to V4A and PITd/v, and the phPIT cluster in humans, (2) area TE in the monkey and its homolog LOC and neighboring fusiform regions, and more speculatively, (3) TGv in the monkey and its possible human equivalent, the temporal pole. We attribute to these levels the visual representations of features, partial real-world entities (RWEs), and known, complete RWEs, respectively. Furthermore, we propose that the middle level, TE and its homolog, is organized into three parallel substreams, lower bank STS, dorsal convexity of TE, and ventral convexity of TE, as are their corresponding human regions. These presumably process shape in depth, 2D shape and material properties, respectively, to construct RWE representations.

Differences in neural activation for object-directed grasping in chimpanzees and humans

The human faculty for object-mediated action, including tool use and imitation, exceeds that of even our closest primate relatives and is a key foundation of human cognitive and cultural uniqueness. In humans and macaques, observing object-directed grasping actions activates a network of frontal, parietal, and occipitotemporal brain regions, but differences in human and macaque activation suggest that this system has been a focus of selection in the primate lineage. To study the evolution of this system, we performed functional neuroimaging in humans' closest living relatives, chimpanzees. We compare activations during performance of an object-directed manual grasping action, observation of the same action, and observation of a mimed version of the action that consisted of only movements without results. Performance and observation of the same action activated a distributed frontoparietal network similar to that reported in macaques and humans. Like humans and unlike macaques, these regions were also activated by observing movements without results. However, in a direct chimpanzee/human comparison, we also identified unique aspects of human neural responses to observed grasping. Chimpanzee activation showed a prefrontal bias, including significantly more activity in ventrolateral prefrontal cortex, whereas human activation was more evenly distributed across more posterior regions, including significantly more activation in ventral premotor cortex, inferior parietal cortex, and inferotemporal cortex. This indicates a more "bottom-up" representation of observed action in the human brain and suggests that the evolution of tool use, social learning, and cumulative culture may have involved modifications of frontoparietal interactions.

Audio-visual perception of 3D cinematography: an fMRI study using condition-based and computation-based analyses

The use of naturalistic stimuli to probe sensory functions in the human brain is gaining increasing interest. Previous imaging studies examined brain activity associated with the processing of cinematographic material using both standard "condition-based" designs, as well as "computational" methods based on the extraction of time-varying features of the stimuli (e.g. motion). Here, we exploited both approaches to investigate the neural correlates of complex visual and auditory spatial signals in cinematography. In the first experiment, the participants watched a piece of a commercial movie presented in four blocked conditions: 3D vision with surround sounds (3D-Surround), 3D with monaural sound (3D-Mono), 2D-Surround, and 2D-Mono. In the second experiment, they watched two different segments of the movie both presented continuously in 3D-Surround. The blocked presentation served for standard condition-based analyses, while all datasets were submitted to computation-based analyses. The latter assessed where activity co-varied with visual disparity signals and the complexity of auditory multi-sources signals. The blocked analyses associated 3D viewing with the activation of the dorsal and lateral occipital cortex and superior parietal lobule, while the surround sounds activated the superior and middle temporal gyri (S/MTG). The computation-based analyses revealed the effects of absolute disparity in dorsal occipital and posterior parietal cortices and of disparity gradients in the posterior middle temporal gyrus plus the inferior frontal gyrus. The complexity of the surround sounds was associated with activity in specific sub-regions of S/MTG, even after accounting for changes of sound intensity. These results demonstrate that the processing of naturalistic audio-visual signals entails an extensive set of visual and auditory areas, and that computation-based analyses can track the contribution of complex spatial aspects characterizing such life-like stimuli.

Physiologically inspired model for the visual recognition of transitive hand actions

The visual recognition of actions is an important visual function that is critical for motor learning and social communication. Action-selective neurons have been found in different cortical regions, including the superior temporal sulcus, parietal and premotor cortex. Among those are mirror neurons, which link visual and motor representations of body movements. While numerous theoretical models for the mirror neuron system have been proposed, the computational basis of the visual processing of goal-directed actions remains largely unclear. While most existing models focus on the possible role of motor representations in action recognition, we propose a model showing that many critical properties of action-selective visual neurons can be accounted for by well-established visual mechanisms. Our model accomplishes the recognition of hand actions from real video stimuli, exploiting exclusively mechanisms that can be implemented in a biologically plausible way by cortical neurons. We show that the model provides a unifying quantitatively consistent account of a variety of electrophysiological results from action-selective visual neurons. In addition, it makes a number of predictions, some of which could be confirmed in recent electrophysiological experiments.

Three-dimensional Shape Coding in Grasping Circuits: A Comparison between the Anterior Intraparietal Area and Ventral Premotor Area F5a

Depth information is necessary for adjusting the hand to the three-dimensional (3-D) shape of an object to grasp it. The transformation of visual information into appropriate distal motor commands is critically dependent on the anterior intraparietal area (AIP) and the ventral premotor cortex (area F5), particularly the F5p sector. Recent studies have demonstrated that both AIP and the F5a sector of the ventral premotor cortex contain neurons that respond selectively to disparity-defined 3-D shape. To investigate the neural coding of 3-D shape and the behavioral role of 3-D shape-selective neurons in these two areas, we recorded single-cell activity in AIP and F5a during passive fixation of curved surfaces and during grasping of real-world objects. Similar to those in AIP, F5a neurons were either first- or second-order disparity selective, frequently showed selectivity for discrete approximations of smoothly curved surfaces that contained disparity discontinuities, and exhibited mostly monotonic tuning for the degree of disparity variation. Furthermore, in both areas, 3-D shape-selective neurons were colocalized with neurons that were active during grasping of real-world objects. Thus, area AIP and F5a contain highly similar representations of 3-D shape, which is consistent with the proposed transfer of object information from AIP to the motor system through the ventral premotor cortex.

Selectivity for three-dimensional shape and grasping-related activity in the macaque ventral premotor cortex

Anatomical studies indicate that area F5 in the macaque ventral premotor cortex consists of three different sectors. One of these is F5a in the posterior bank of the inferior arcuate sulcus, but no functional characterization of F5a at the single-cell level exists. We investigated the neuronal selectivity for three-dimensional (3D) shape and grasping activity in F5a. In contrast to neighboring regions F5p and 45B, the great majority of F5a neurons showed selectivity for disparity-defined curved surfaces, and most neurons preserved this selectivity across positions in depth, indicating higher-order disparity selectivity. Thus, as predicted by monkey fMRI data, F5a neurons showed robust 3D-shape selectivity in the absence of a motor response. To investigate the relationship between disparity selectivity and grasping activity, we recorded from 3D-shape-selective F5a neurons during a visually guided grasping task and during grasping in the dark. F5a neurons encoding the depth profile of curved surfaces frequently responded during grasping of real-world objects in the light, but not in the dark, whereas nearby neurons were also active in the dark. The presence of 3D-shape-selective and "visual-dominant" neurons demonstrates that the F5a sector is distinct from neighboring regions of ventral premotor cortex, in line with recent anatomical connectivity studies.

Neural selectivity and representation of gloss in the monkey inferior temporal cortex

When we view an object, its appearance depends in large part on specific surface reflectance properties; among these is surface gloss, which provides important information about the material composition of the object and the fine structure of its surface. To study how gloss is represented in the visual cortical areas related to object recognition, we examined the responses of neurons in the inferior temporal (IT) cortex of the macaque monkey to a set of object images exhibiting various combinations of specular reflection, diffuse reflection, and roughness, which are important physical parameters of surface gloss. We found that there are neurons in the lower bank of the superior temporal sulcus that selectively respond to specific gloss. This neuronal selectivity was largely maintained when the shape or illumination of the object was modified and perceived glossiness was unchanged. By contrast, neural responses were significantly altered when the pixels of the images were randomly rearranged, and perceived glossiness was dramatically changed. The stimulus preference of these neurons differed from cell to cell, and, as a population, they systematically represented a variety of surface glosses. We conclude that, within the visual cortex, there are mechanisms operating to integrate local image features and extract information about surface gloss and that this information is systematically represented in the IT cortex, an area playing an important role in object recognition.

Inferotemporal cortex subserves three-dimensional structure categorization

We perceive real-world objects as three-dimensional (3D), yet it is unknown which brain area underlies our ability to perceive objects in this way. The macaque inferotemporal (IT) cortex contains neurons that respond selectively to 3D structures defined by binocular disparity. To examine the causal role of IT in the categorization of 3D structures, we electrically stimulated clusters of IT neurons with a similar 3D-structure preference while monkeys performed a 3D-structure categorization task. Microstimulation of 3D-structure-selective IT clusters caused monkeys to choose the preferred structure of the 3D-structure-selective neurons considerably more often. Microstimulation in IT also accelerated the monkeys' choice for the preferred structure, while delaying choices corresponding to the nonpreferred structure of a given site. These findings reveal that 3D-structure-selective neurons in IT contribute to the categorization of 3D objects.

The extraction of 3D shape in the visual system of human and nonhuman primates

Depth structure, the third dimension of object shape, is extracted from disparity, motion, texture, and shading in the optic array. Gradient-selective neurons play a key role in this process. Such neurons occur in CIP, AIP, TEs, and F5 (for first- or second-order disparity gradients), in MT/V5, in FST (for speed gradients), and in CIP and TEs (for texture gradients). Most of these regions are activated during magnetic resonance scanning in alert monkeys by comparing 3D conditions with the 2D controls for the different cues. Similarities in activation patterns of monkeys and humans tested with identical paradigms suggest that like gradient-selective neurons are found in corresponding human cortical areas. This view gains credence as the homologies between such areas become more evident. Furthermore, 3D shape-processing networks are similar in the two species, with the exception of the greater involvement of human posterior parietal cortex in the extraction of 3D shape from motion. Thus we can begin to understand how depth structure is extracted from motion, disparity, and texture in the primate brain, but the extraction of depth structure from shading and that of wire-like objects requires further scrutiny.

Motor planning is facilitated by adopting an action's goal posture: an fMRI study

Motor planning is a hierarchical process that is typically organized around an action's goal (e.g., drinking from a cup). However, the motor plan depends not only on the goal but also on the current body state. Here, we investigated how one's own body posture interacts with planning of goal-directed actions. Participants engaged in a grasp selection (GS) task while we manipulated their arm posture. They had to indicate how they would grasp a bar when transporting it from a start to goal position and orientation. We compared situations in which one's body posture was in-congruent with the start posture and/or goal posture of the planned movement. Behavioral results show that GS took longer when one's own body state was incongruent with the goal posture of the planned movement. Correspondingly, neural activity in the intraparietal sulcus (IPS) and extrastriate body area (EBA) was modulated by congruency between the body state and the action plan. IPS was sensitive to overall congruency between body posture and action plan, while the EBA was sensitive specifically to goal posture congruency. Together, our results suggest that IPS maintains an internal state of one's own body posture, while EBA contains a representation of the goal posture of the action plan.

The selectivity of neurons in the macaque fundus of the superior temporal area for three-dimensional structure from motion

Motion is a potent cue for the perception of three-dimensional (3D) shape in primates, but little is known about its underlying neural mechanisms. Guided by recent functional magnetic resonance imaging results, we tested neurons in the fundus of the superior temporal sulcus (FST) area of two macaque monkeys (Macaca mulatta, one male) using motion-defined surface patches with various 3D shapes such as slanted planes, saddles, or cylinders. The majority of the FST neurons (>80%) were selective for stimuli depicting specific shapes, and all the surfaces tested were represented among the selective FST neurons. Importantly, this selectivity tolerated changes in speed, position, size, or between binocular and monocular presentations. This tolerance demonstrates that the 3D structure-from-motion (3D-SFM) selectivity of FST neurons is a higher-order selectivity, which cannot be reduced to a lower-order speed selectivity. The 3D-SFM selectivity of FST neurons was unaffected by removal of the opposed-motion cue that supplemented the speed gradient cue in the standard stimuli. When tested with the same standard stimuli, fewer neurons in the middle temporal/visual 5 (MT/V5) area were selective than FST neurons. In addition, selective MT/V5 neurons represented fewer types of surfaces and were less tolerant of stimulus changes than FST neurons. Overall, these results indicate that FST neurons code motion-defined 3D shape fragments, underscoring the central role of FST in processing 3D-SFM.

Cortical connections of the anterior (F5a) subdivision of the macaque ventral premotor area F5

We traced the cortical connections of the anterior sector (F5a) of the macaque ventral premotor (PMv) area F5 and compared them with those of the adjacent F5 sectors, F5c and F5p. F5a displays a very dense "intrinsic" connectivity with F5c and F5p, premotor connections limited to F4 and F6/pre-SMA, relatively robust prefrontal connections with areas 46v and 12, and dense connections with rostral opercular frontal areas. Outside the frontal cortex, connections of F5a are dense with the SII region, relatively robust with inferior parietal areas PFG and AIP, weak with the inferior parietal area PF, and moderate with area 24. The comparison with data from injections in F5c and F5p showed that F5a, though sharing some common parietal connections with the other F5 sectors, displays several characterizing features providing robust evidence for its connectional distinctiveness. The present study provides evidence for a general organization of the PMv similar to that of the medial and dorsal premotor cortex, with F5a representing a pre-PMv area. Specifically, the present data suggest that F5a is a privileged site of integration, in the PMv, of parietal sensory-motor signals with higher-order information originating from prefrontal, rostral frontal opercular areas, and F6/pre-SMA. The results of this integration can be then broadcasted to the adjacent F5 sectors for the generation and control of hand actions and cognitive motor functions.

The cognitive neuroscience of prehension: recent developments

Prehension, the capacity to reach and grasp, is the key behavior that allows humans to change their environment. It continues to serve as a remarkable experimental test case for probing the cognitive architecture of goal-oriented action. This review focuses on recent experimental evidence that enhances or modifies how we might conceptualize the neural substrates of prehension. Emphasis is placed on studies that consider how precision grasps are selected and transformed into motor commands. Then, the mechanisms that extract action relevant information from vision and touch are considered. These include consideration of how parallel perceptual networks within parietal cortex, along with the ventral stream, are connected and share information to achieve common motor goals. On-line control of grasping action is discussed within a state estimation framework. The review ends with a consideration about how prehension fits within larger action repertoires that solve more complex goals and the possible cortical architectures needed to organize these actions.