Areal differences in depth cue integration between monkey and human

Single-Trial fMRI Decoding of 3D Motion with Stereoscopic and Perspective Cues

How does the brain process 3D motion? We focused on the human motion complex (hMT+), extending insights from monkey studies. Using 3D-motion stimuli containing perspective and/or stereoscopic cues, we investigated the hierarchy within the motion complex in humans of both sexes to understand the neural mechanisms underlying motion perception. On each trial we decoded 3D motion direction (toward/away) based on the BOLD response in primary visual cortex (V1), and regions within hMT+ including the middle temporal (MT) and medial superior temporal (MST) areas, and the fundus of the superior temporal sulcus (FST). We found that 3D-motion direction could be reliably decoded from all four areas but accuracy depended on cue content. MT and FST showed greatest decoding accuracy with perspective and stereoscopic cues, respectively. Decoding of motion direction in V1 and MST could be explained by retinotopic biases in the BOLD response to motion stimuli. MT and FST were less impacted by such biases. We also identified significant behavioral differences between participants: some were proficient at using stereoscopic cues and others performed near chance. Good behavioral performance with stereoscopic cues was accompanied by better decoding performance in FST but not MT. A control experiment that eliminated 3D motion percepts for stereoscopic stimuli, but not perspective stimuli, revealed that unlike MT, decoding accuracy in FST was influenced by perceptual components of 3D motion. Our findings support that MT and FST play distinct roles in the analysis of visual motion and are key in the transformation of retinal input into perceptual report.

Functional Localization of Visual Motion Area FST in Humans

The fundus of the superior temporal sulcus (FST) in macaque monkeys is implicated in the processing of complex motion signals, yet a human homolog remains elusive. To better understand the neural mechanisms underlying the analysis of complex motion signals in humans, it is crucial to understand if and where a region homologous to FST exists. Here, we considered potential localizers and evaluated their effectiveness in delineating putative FST (pFST) from two nearby motion-sensitive areas, hMT and MST, in humans. Nine participants underwent fMRI scanning sessions with 2D and 3D motion localizers, as well as population receptive field (pRF) mapping. We observed consistent anterior and inferior activation relative to hMT and MST in response to stimuli that contained coherent 3D, but not 2D, motion. Motion opponency and myelination measures further validated the functional and structural distinction between pFST and hMT/MST. At the same time, standard pRF mapping techniques that reveal the visual field organization of hMT/MST proved suboptimal for delineating pFST. Our findings provide confirmatory evidence for the existence of a functional homolog of macaque area FST in humans, offer a robust framework for localizing pFST, and underscore the area's distinct role in visual motion processing.

Identifying cortical areas that underlie the transformation from 2D retinal to 3D head-centric motion signals

Accurate motion perception requires that the visual system integrate the 2D retinal motion signals received by the two eyes into a single representation of 3D motion. However, most experimental paradigms present the same stimulus to the two eyes, signaling motion limited to a 2D fronto-parallel plane. Such paradigms are unable to dissociate the representation of 3D head-centric motion signals (i.e., 3D object motion relative to the observer) from the associated 2D retinal motion signals. Here, we used stereoscopic displays to present separate motion signals to the two eyes and examined their representation in visual cortex using fMRI. Specifically, we presented random-dot motion stimuli that specified various 3D head-centric motion directions. We also presented control stimuli, which matched the motion energy of the retinal signals, but were inconsistent with any 3D motion direction. We decoded motion direction from BOLD activity using a probabilistic decoding algorithm. We found that 3D motion direction signals can be reliably decoded in three major clusters in the human visual system. Critically, in early visual cortex (V1-V3), we found no significant difference in decoding performance between stimuli specifying 3D motion directions and the control stimuli, suggesting that these areas represent the 2D retinal motion signals, rather than 3D head-centric motion itself. In voxels in and surrounding hMT and IPS0 however, decoding performance was consistently superior for stimuli that specified 3D motion directions compared to control stimuli. Our results reveal the parts of the visual processing hierarchy that are critical for the transformation of retinal into 3D head-centric motion signals and suggest a role for IPS0 in their representation, in addition to its sensitivity to 3D object structure and static depth.

Disparity in Context: Understanding how monocular image content interacts with disparity processing in human visual cortex

Horizontal disparities between the two eyes' retinal images are the primary cue for depth. Commonly used random ot tereograms (RDS) intentionally camouflage the disparity cue, breaking the correlations between monocular image structure and the depth map that are present in natural images. Because of the nonlinear nature of visual processing, it is unlikely that simple computational rules derived from RDS will be sufficient to explain binocular vision in natural environments. In order to understand the interplay between natural scene structure and disparity encoding, we used a depth-image-based-rendering technique and a library of natural 3D stereo pairs to synthesize two novel stereogram types in which monocular scene content was manipulated independent of scene depth information. The half-images of the novel stereograms comprised either random-dots or scrambled natural scenes, each with the same depth maps as the corresponding natural scene stereograms. Using these stereograms in a simultaneous Event-Related Potential and behavioral discrimination task, we identified multiple disparity-contingent encoding stages between 100 ~ 500 msec. The first disparity sensitive evoked potential was observed at ~100 msec after an earlier evoked potential (between ~50-100 msec) that was sensitive to the structure of the monocular half-images but blind to disparity. Starting at ~150 msec, disparity responses were stereogram-specific and predictive of perceptual depth. Complex features associated with natural scene content are thus at least partially coded prior to disparity information, but these features and possibly others associated with natural scene content interact with disparity information only after an intermediate, 2D scene-independent disparity processing stage.

Common functional localizers to enhance NHP & cross-species neuroscience imaging research

Functional localizers are invaluable as they can help define regions of interest, provide cross-study comparisons, and most importantly, allow for the aggregation and meta-analyses of data across studies and laboratories. To achieve these goals within the non-human primate (NHP) imaging community, there is a pressing need for the use of standardized and validated localizers that can be readily implemented across different groups. The goal of this paper is to provide an overview of the value of localizer protocols to imaging research and we describe a number of commonly used or novel localizers within NHPs, and keys to implement them across studies. As has been shown with the aggregation of resting-state imaging data in the original PRIME-DE submissions, we believe that the field is ready to apply the same initiative for task-based functional localizers in NHP imaging. By coming together to collect large datasets across research group, implementing the same functional localizers, and sharing the localizers and data via PRIME-DE, it is now possible to fully test their robustness, selectivity and specificity. To do this, we reviewed a number of common localizers and we created a repository of well-established localizer that are easily accessible and implemented through the PRIME-RE platform.

Unique Neural Activity Patterns Among Lower Order Cortices and Shared Patterns Among Higher Order Cortices During Processing of Similar Shapes With Different Stimulus Types

We investigated the neural mechanism of the processing of three-dimensional (3D) shapes defined by disparity and perspective. We measured blood oxygenation level-dependent signals as participants viewed and classified 3D images of convex-concave shapes. According to the cue (disparity or perspective) and element type (random dots or black and white dotted lines), three types of stimuli were used: random dot stereogram, black and white dotted lines with perspective, and black and white dotted lines with binocular disparity. The blood oxygenation level-dependent images were then classified by multivoxel pattern analysis. To identify areas selective to shape, we assessed convex-concave classification accuracy with classifiers trained and tested using signals evoked by the same stimulus type (same cue and element type). To identify cortical regions with similar neural activity patterns regardless of stimulus type, we assessed the convex-concave classification accuracy of transfer classification in which classifiers were trained and tested using different stimulus types (different cues or element types). Classification accuracy using the same stimulus type was high in the early visual areas and subregions of the intraparietal sulcus (IPS), whereas transfer classification accuracy was high in the dorsal subregions of the IPS. These results indicate that the early visual areas process the specific features of stimuli, whereas the IPS regions perform more generalized processing of 3D shapes, independent of a specific stimulus type.

Rapid event-related, BOLD fMRI, non-human primates (NHP): choose two out of three

Human functional magnetic resonance imaging (fMRI) typically employs the blood-oxygen-level-dependent (BOLD) contrast mechanism. In non-human primates (NHP), contrast enhancement is possible using monocrystalline iron-oxide nanoparticles (MION) contrast agent, which has a more temporally extended response function. However, using BOLD fMRI in NHP is desirable for interspecies comparison, and the BOLD signal’s faster response function promises to be beneficial for rapid event-related (rER) designs. Here, we used rER BOLD fMRI in macaque monkeys while viewing real-world images, and found visual responses and category selectivity consistent with previous studies. However, activity estimates were very noisy, suggesting that the lower contrast-to-noise ratio of BOLD, suboptimal behavioural performance, and motion artefacts, in combination, render rER BOLD fMRI challenging in NHP. Previous studies have shown that rER fMRI is possible in macaques with MION, despite MION’s prolonged response function. To understand this, we conducted simulations of the BOLD and MION response during rER, and found that no matter how fast the design, the greater amplitude of the MION response outweighs the contrast loss caused by greater temporal smoothing. We conclude that although any two of the three elements (rER, BOLD, NHP) have been shown to work well, the combination of all three is particularly challenging.