Feature integration and object representations along the dorsal stream visual hierarchy

Position- and scale-invariant object-centered spatial localization in monkey frontoparietal cortex dynamically adapts to cognitive demand

Egocentric encoding is a well-known property of brain areas along the dorsal pathway. Different to previous experiments, which typically only demanded egocentric spatial processing during movement preparation, we designed a task where two male rhesus monkeys memorized an on-the-object target position and then planned a reach to this position after the object re-occurred at variable location with potentially different size. We found allocentric (in addition to egocentric) encoding in the dorsal stream reach planning areas, parietal reach region and dorsal premotor cortex, which is invariant with respect to the position, and, remarkably, also the size of the object. The dynamic adjustment from predominantly allocentric encoding during visual memory to predominantly egocentric during reach planning in the same brain areas and often the same neurons, suggests that the prevailing frame of reference is less a question of brain area or processing stream, but more of the cognitive demands.

Distinct spatial maps and multiple object codes in the lateral entorhinal cortex

The lateral entorhinal cortex (LEC) is a major cortical input area to the hippocampus, and it is crucial for associative object-place-context memories. An unresolved question is whether these associations are performed exclusively in the hippocampus or also upstream of it. Anatomical evidence suggests that the LEC processes both object and spatial information. We describe here a gradient of spatial selectivity along the antero-posterior axis of the LEC. We demonstrate that the LEC generates distinct spatial maps for different contexts that are independent of object coding and vice versa, thus providing evidence for pure spatial and pure object codes upstream of the hippocampus. While space and object coding occur by and large separately in the LEC, we identified neurons that encode for space and objects conjunctively. Together, these findings point to a scenario in which the LEC sustains both distinct space and object coding and associative space-object coding.

Identifying critical kinematic features of animate motion and contribution to animacy perception

Humans can distinguish flying birds from drones based solely on motion features when no image information is available. However, it remains unclear which motion features of animate motion induce our animacy perception. To address this, we first analyzed the differences in centroid motion between birds and drones, and discovered that birds exhibit greater acceleration, angular speed, and trajectory fluctuations. We further determined the order of their importance in evoking animacy perception was trajectory fluctuations, acceleration, and speed. More interestingly, people judge whether a moving object is alive using a feature-matching strategy, implying that animacy perception is induced in a key feature-triggered way rather than relying on the accumulation of evidence. Our findings not only shed light on the critical motion features that induce animacy perception and their relative contributions but also have important implications for developing target classification algorithms based on motion features.

Episodic Memory formation: A review of complex Hippocampus input pathways

Memories of everyday experiences involve the encoding of a rich and dynamic representation of present objects and their contextual features. Traditionally, the resulting mnemonic trace is referred to as Episodic Memory, i.e. the "what", "where" and "when" of a lived episode. The journey for such memory trace encoding begins with the perceptual data of an experienced episode handled in sensory brain regions. The information is then streamed to cortical areas located in the ventral Medio Temporal Lobe, which produces multi-modal representations concerning either the objects (in the Perirhinal cortex) or the spatial and contextual features (in the parahippocampal region) of the episode. Then, this high-level data is gated through the Entorhinal Cortex and forwarded to the Hippocampal Formation, where all the pieces get bound together. Eventually, the resulting encoded neural pattern is relayed back to the Neocortex for a stable consolidation. This review will detail these different stages and provide a systematic overview of the major cortical streams toward the Hippocampus relevant for Episodic Memory encoding.

Visuo-haptic object perception for robots: an overview

The object perception capabilities of humans are impressive, and this becomes even more evident when trying to develop solutions with a similar proficiency in autonomous robots. While there have been notable advancements in the technologies for artificial vision and touch, the effective integration of these two sensory modalities in robotic applications still needs to be improved, and several open challenges exist. Taking inspiration from how humans combine visual and haptic perception to perceive object properties and drive the execution of manual tasks, this article summarises the current state of the art of visuo-haptic object perception in robots. Firstly, the biological basis of human multimodal object perception is outlined. Then, the latest advances in sensing technologies and data collection strategies for robots are discussed. Next, an overview of the main computational techniques is presented, highlighting the main challenges of multimodal machine learning and presenting a few representative articles in the areas of robotic object recognition, peripersonal space representation and manipulation. Finally, informed by the latest advancements and open challenges, this article outlines promising new research directions.

Visual processing deficits in patients with schizophrenia spectrum and bipolar disorders and associations with psychotic symptoms, and intellectual abilities

Objective

Low-level sensory disruption is hypothesized as a precursor to clinical and cognitive symptoms in severe mental disorders. We compared visual discrimination performance in patients with schizophrenia spectrum disorder or bipolar disorder with healthy controls, and investigated associations with clinical symptoms and IQ.

Methods

Patients with schizophrenia spectrum disorder (n = 32), bipolar disorder (n = 55) and healthy controls (n = 152) completed a computerized visual discrimination task. Participants responded whether the latter of two consecutive grids had higher or lower spatial frequency, and discrimination thresholds were estimated using an adaptive maximum likelihood procedure. Case-control differences in threshold were assessed using linear regression, F-test and post-hoc pair-wise comparisons. Linear models were used to test for associations between visual discrimination threshold and psychotic symptoms derived from the PANSS and IQ assessed using the Matrix Reasoning and Vocabulary subtests from the Wechsler Abbreviated Scale of Intelligence (WASI).

Results

Robust regression revealed a significant main effect of diagnosis on discrimination threshold (robust F = 6.76, p = .001). Post-hoc comparisons revealed that patients with a schizophrenia spectrum disorder (mean = 14%, SD = 0.08) had higher thresholds compared to healthy controls (mean = 10.8%, SD = 0.07, β = 0.35, t = 3.4, p = .002), as did patients with bipolar disorder (12.23%, SD = 0.07, β = 0.21, t = 2.42, p = .04). There was no significant difference between bipolar disorder and schizophrenia (β = -0.14, t = -1.2, p = .45). Linear models revealed negative associations between IQ and threshold across all participants when controlling for diagnostic group (β = -0.3, t = -3.43, p = .0007). This association was found within healthy controls (t = -3.72, p = .0003) and patients with bipolar disorder (t = -2.53, p = .015), and no significant group by IQ interaction on threshold (F = 0.044, p = .97). There were no significant associations between PANSS domain scores and discrimination threshold.

Conclusion

Patients with schizophrenia spectrum or bipolar disorders exhibited higher visual discrimination thresholds than healthy controls, supporting early visual deficits among patients with severe mental illness. Discrimination threshold was negatively associated with IQ among healthy controls and bipolar disorder patients. These findings elucidate perception-related disease mechanisms in severe mental illness, which warrants replication in independent samples.

Magnocellular and parvocellular contributions to brain network dysfunction during learning and memory: Implications for schizophrenia

Memory deficits are core features of schizophrenia, and a central aim in biological psychiatry is to identify the etiology of these deficits. Scrutiny is naturally focused on the dorsolateral prefrontal cortex and the hippocampal cortices, given these structures' roles in memory and learning. The fronto-hippocampal framework is valuable but restrictive. Network-based underpinnings of learning and memory are substantially diverse and include interactions between hetero-modal and early sensory networks. Thus, a loss of fidelity in sensory information may impact memorial and cognitive processing in higher-order brain sub-networks, becoming a sensory source for learning and memory deficits. In this overview, we suggest that impairments in magno- and parvo-cellular visual pathways result in degraded inputs to core learning and memory networks. The ascending cascade of aberrant neural events significantly contributes to learning and memory deficits in schizophrenia. We outline the network bases of these effects, and suggest that any network perspectives of dysfunction in schizophrenia must assess the impact of impaired perceptual contributions. Finally, we speculate on how this framework enriches the space of biomarkers and expands intervention strategies to ameliorate this prototypical disconnection syndrome.

Current directions in visual perceptual learning

The visual expertise of adult humans is jointly determined by evolution, visual development, and visual perceptual learning. Perceptual learning refers to performance improvements in perceptual tasks after practice or training in the task. It occurs in almost all visual tasks, ranging from simple feature detection to complex scene analysis. In this Review, we focus on key behavioral aspects of visual perceptual learning. We begin by describing visual perceptual learning tasks and manipulations that influence the magnitude of learning, and then discuss specificity of learning. Next, we present theories and computational models of learning and specificity. We then review applications of visual perceptual learning in visual rehabilitation. Finally, we summarize the general principles of visual perceptual learning, discuss the tension between plasticity and stability, and conclude with new research directions.

Attention to visual motion suppresses neuronal and behavioral sensitivity in nearby feature space

Background

Feature-based attention prioritizes the processing of the attended feature while strongly suppressing the processing of nearby ones. This creates a non-linearity or “attentional suppressive surround” predicted by the Selective Tuning model of visual attention. However, previously reported effects of feature-based attention on neuronal responses are linear, e.g., feature-similarity gain. Here, we investigated this apparent contradiction by neurophysiological and psychophysical approaches.

Results

Responses of motion direction-selective neurons in area MT/MST of monkeys were recorded during a motion task. When attention was allocated to a stimulus moving in the neurons’ preferred direction, response tuning curves showed its minimum for directions 60–90° away from the preferred direction, an attentional suppressive surround. This effect was modeled via the interaction of two Gaussian fields representing excitatory narrowly tuned and inhibitory widely tuned inputs into a neuron, with feature-based attention predominantly increasing the gain of inhibitory inputs. We further showed using a motion repulsion paradigm in humans that feature-based attention produces a similar non-linearity on motion discrimination performance.

Conclusions

Our results link the gain modulation of neuronal inputs and tuning curves examined through the feature-similarity gain lens to the attentional impact on neural population responses predicted by the Selective Tuning model, providing a unified framework for the documented effects of feature-based attention on neuronal responses and behavior.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12915-022-01428-7.

Laminar functional magnetic resonance imaging in vision research

Magnetic resonance imaging (MRI) scanners at ultra-high magnetic fields have become available to use in humans, thus enabling researchers to investigate the human brain in detail. By increasing the spatial resolution, ultra-high field MR allows both structural and functional characterization of cortical layers. Techniques that can differentiate cortical layers, such as histological studies and electrode-based measurements have made critical contributions to the understanding of brain function, but these techniques are invasive and thus mainly available in animal models. There are likely to be differences in the organization of circuits between humans and even our closest evolutionary neighbors. Thus research on the human brain is essential. Ultra-high field MRI can observe differences between cortical layers, but is non-invasive and can be used in humans. Extensive previous literature has shown that neuronal connections between brain areas that transmit feedback and feedforward information terminate in different layers of the cortex. Layer-specific functional MRI (fMRI) allows the identification of layer-specific hemodynamic responses, distinguishing feedback and feedforward pathways. This capability has been particularly important for understanding visual processing, as it has allowed researchers to test hypotheses concerning feedback and feedforward information in visual cortical areas. In this review, we provide a general overview of successful ultra-high field MRI applications in vision research as examples of future research.

The role of temporal cortex in the control of attention

Attention is an indispensable component of active vision. Contrary to the widely accepted notion that temporal cortex processing primarily focusses on passive object recognition, a series of very recent studies emphasize the role of temporal cortex structures, specifically the superior temporal sulcus (STS) and inferotemporal (IT) cortex, in guiding attention and implementing cognitive programs relevant for behavioral tasks. The goal of this theoretical paper is to advance the hypothesis that the temporal cortex attention network (TAN) entails necessary components to actively participate in attentional control in a flexible task-dependent manner. First, we will briefly discuss the general architecture of the temporal cortex with a focus on the STS and IT cortex of monkeys and their modulation with attention. Then we will review evidence from behavioral and neurophysiological studies that support their guidance of attention in the presence of cognitive control signals. Next, we propose a mechanistic framework for executive control of attention in the temporal cortex. Finally, we summarize the role of temporal cortex in implementing cognitive programs and discuss how they contribute to the dynamic nature of visual attention to ensure flexible behavior.

Enhanced Information Flow From Cerebellum to Secondary Visual Cortices Leads to Better Surgery Outcome in Degenerative Cervical Myelopathy Patients: A Stochastic Dynamic Causal Modeling Study With Functional Magnetic Resonance Imaging

Degenerative cervical myelopathy (DCM) damages the spinal cord, resulting in long-term neurological impairment including motor and visual deficits. Given that visual feedback is crucial in guiding movements, the visual disorder may be a cause of motor deficits in patients with DCM. It has been shown that increased functional connectivity between secondary visual cortices and cerebellum, which are functionally related to the visually guided movements, was correlated with motor function in patients with DCM. One possible explanation is that the information integration between these regions was increased to compensate for impaired visual acuity in patients with DCM and resulted in better visual feedback during motor function. However, direct evidence supporting this hypothesis is lacking. To test this hypothesis and explore in more detail the information flow within the "visual-cerebellum" system, we measured the effective connectivity (EC) among the "visual-cerebellum" system via dynamic causal modeling and then tested the relationship between the EC and visual ability in patients with DCM. Furthermore, the multivariate pattern analysis was performed to detect the relationship between the pattern of EC and motor function in patients with DCM. We found (1) significant increases of the bidirectional connections between bilateral secondary visual cortices and cerebellum were observed in patients with DCM; (2) the increased self-connection of the cerebellum was positively correlated with the impaired visual acuity in patients; (3) the amplitude of effectivity from the cerebellum to secondary visual cortices was positively correlated with better visual recovery following spinal cord decompression surgery; and (4) the pattern of EC among the visual-cerebellum system could be used to predict the pre-operative motor function. In conclusion, this study provided direct evidence that the increased information integration within the "visual-cerebellum" system compensated for visual impairments, which might have importance for sustaining better motor function in patients with DCM.

Colors and Handles: How Action Primes Perception

How deeply does action influence perception? Does action performance affect the perception of object features directly related to action only? Or does it concern also object features such as colors, which are not held to directly afford action? The present study aimed at answering these questions. We asked participants to repeatedly grasp a handled mug hidden from their view before judging whether a visually presented mug was blue rather than cyan. The motor training impacted on their perceptual judgments, by speeding participants' responses, when the handle of the presented mug was spatially aligned with the trained hand. The priming effect did not occur when participants were trained to merely touch the mug with their hand closed in a fist. This indicates that action performance may shape the perceptual judgment on object features, even when these features are colors and do not afford any action. How we act on surrounding objects is therefore not without consequence for how we experience them.

Vision, attention, and driving

Safe driving demands the coordination of multiple sensory and cognitive functions, such as vision and attention. Patients with neurologic or ophthalmic disease are exposed to selective pathophysiologic insults to driving-critical systems, placing them at a higher risk for unsafe driving and restricted driving privileges. Here, we evaluate how vision and attention contribute to unsafe driving across different patient populations. In ophthalmic disease, we focus on macular degeneration, glaucoma, diabetic retinopathy, and cataract; in neurologic disease, we focus on Alzheimer's disease, Parkinson's disease, and multiple sclerosis. Unsafe driving is generally associated with impaired vision and attention in ophthalmic and neurologic patients, respectively. Furthermore, patients with ophthalmic disease experience some degree of impairment in attention. Similarly, patients with neurologic disease experience some degree of impairment in vision. While numerous studies have demonstrated a relationship between impaired vision and unsafe driving in neurologic disease, there remains a dearth of knowledge regarding the relationship between impaired attention and unsafe driving in ophthalmic disease. In summary, this chapter confirms-and offers opportunities for future research into-the contribution of vision and attention to safe driving.

Development of the Parental Questionnaire for Cerebral Visual Impairment in Children Younger than 72 Months

Background and Purpose

Cerebral visual impairment (CVI) is an underdiagnosed condition in children, and its assessment tools have focused on older children. We aimed to develop a parental questionnaire for cerebral visual impairment (PQCVI) for screening CVI in young children.

Methods

The PQCVI comprised 23 questions based on a modified version of Houliston and Dutton's questionnaire for older children. The PQCVI with neurocognitive function tests was applied to 201 child-parent pairs with typically developing children younger than 72 months (age 32.4±20.1 months, mean±standard deviation). The children were classified into six age groups. The normative data, cutoff scores, and internal reliability were assessed and item analysis was performed. We referred to the total score for all questions as the cerebral visual function (CVF) score.

Results

The normative data showed that the CVF score and the scores corresponding to ventral-stream and dorsal-stream visual functions plausibly increased with age. The scores rapidly reached 90% of their maximum values up to the age of 36 months, after which they increased slowly. Cronbach's alpha for all questions across all age groups was 0.97, showing excellent consistency. The item difficulty and item discrimination coefficients showed that the questions were generally adequate for this age stage.

Conclusions

The PQCVI items produced reliable responses in children younger than 72 months. The rapid increase in scores before the age of 3 years supports the importance of early identification of CVI. Following additional clinical verification, the PQCVI may be useful for CVI screening.

Gilles de la Tourette Syndrome-A Disorder of Action-Perception Integration

Gilles de la Tourette syndrome is a multifaceted and complex neuropsychiatric disorder. Given that tics as motor phenomena are the defining and cardinal feature of Tourette syndrome, it has long been conceptualized as a motor/movement disorder. However, considering premonitory urges preceding tics, hypersensitivity to external stimuli and abnormalities in sensorimotor integration perceptual processes also seem to be relevant in the pathophysiology of Tourette syndrome. In addition, tic expression depends on attention and tics can, at least partly and transiently, be controlled, so that cognitive processes need to be considered as well. Against this background, explanatory concepts should encompass not only the motor phenomenon tic but also perceptual and cognitive processes. Representing a comprehensive theory of the processing of perceptions and actions paying particular attention to their interdependency and the role of cognitive control, the Theory of Event Coding seems to be a suitable conceptual framework for the understanding of Tourette syndrome. In fact, recent data suggests that addressing the relation between actions (i.e., tics) and perceptions (i.e., sensory phenomena like premonitory urges) in the context of event coding allows to gaining relevant insights into perception-action coding in Tourette syndrome indicating that perception action binding is abnormally strong in this disorder.

Evidence of an Effect of Gaming Experience on Visuospatial Attention in Deaf but Not in Hearing Individuals

Auditory cortex in congenitally deaf early sign language users reorganizes to support cognitive processing in the visual domain. However, evidence suggests that the potential benefits of this reorganization are largely unrealized. At the same time, there is growing evidence that experience of playing computer and console games improves visual cognition, in particular visuospatial attentional processes. In the present study, we investigated in a group of deaf early signers whether those who reported recently playing computer or console games (deaf gamers) had better visuospatial attentional control than those who reported not playing such games (deaf non-gamers), and whether any such effect was related to cognitive processing in the visual domain. Using a classic test of attentional control, the Eriksen Flanker task, we found that deaf gamers performed on a par with hearing controls, while the performance of deaf non-gamers was poorer. Among hearing controls there was no effect of gaming. This suggests that deaf gamers may have better visuospatial attentional control than deaf non-gamers, probably because they are less susceptible to parafoveal distractions. Future work should examine the robustness of this potential gaming benefit and whether it is associated with neural plasticity in early deaf signers, as well as whether gaming intervention can improve visuospatial cognition in deaf people.

Investigating the Interaction Between Form and Motion Processing: A Review of Basic Research and Clinical Evidence

A widely held view of the visual system supported the perspective that the primate brain is organized in two main specialized streams, called the ventral and dorsal streams. The ventral stream is known to be involved in object recognition (e.g., form and orientation). In contrast, the dorsal stream is thought to be more involved in spatial recognition (e.g., the spatial relationship between objects and motion direction). Recent evidence suggests that these two streams are not segregated but interact with each other. A class of visual stimuli known as Glass patterns has been developed to shed light on this process. Glass patterns are visual stimuli made of pairs of dots, called dipoles, that give the percept of a specific form or apparent motion, depending on the spatial and temporal arrangement of the dipoles. In this review, we show an update of the neurophysiological, brain imaging, psychophysical, clinical, and brain stimulation studies which have assessed form and motion integration mechanisms, and the level at which this occurs in the human and non-human primate brain. We also discuss several studies based on non-invasive brain stimulation techniques that used different types of visual stimuli to assess the cortico-cortical interactions in the visual cortex for the processing of form and motion information. Additionally, we discuss the timing of specific visual processing in the ventral and dorsal streams. Finally, we report some parallels between healthy participants and neurologically impaired patients in the conscious processing of form and motion.

Sex differences in implicit processing of allocentric relationships between objects and location in a Simon task

Simon tasks reveal implicit processing conflicts that arise when the abstract coding of stimulus position is incongruent with coding for location of the output response. Participants were tested with two versions of a Simon task in a counterbalanced order to examine a potential female bias for attending to object characteristics versus object location. Both tasks used a triangle pointing to the left or right. A simple version presented the triangle in an inner or outer position relative to central fixation. A more complex version included a frame surrounding the inner-outer triangle presentation area in order to introduce additional visual elements for left/right visual processing. When the No Frame version was the first presented, there were no sex differences in the Simon effect in either version, which is consistent with results from other studies that did not provide feedback regarding accuracy. When the initial test was the Frame version, we observed a reverse Simon effect for incongruent triangles presented in the left inner position, with females faster than males to identify the incongruent condition versus the congruent (-59 vs -5 msec). In the No Frame condition that followed, females showed a carryover effect from the previous Frame condition, exhibiting positive Simon effects that were two fold larger than males for identifying incongruent stimuli presented in the left and right outer positions. Similar to previous Simon studies, females showed longer overall reaction times than males (~15%). The difference was not related to the Simon effect and is also found in other types of tasks involving early visual processing of objects with location. Based on sex differences in the Simon effect that emerged following initial experience of the triangle adjoining the frame, the present results support a female bias toward broader integration of objects within the context of location.

Projections between visual cortex and pulvinar in the rat

The extrageniculate visual pathway, which carries visual information from the retina through the superficial layers of the superior colliculus and the pulvinar, is poorly understood. The pulvinar is thought to modulate information flow between cortical areas, and has been implicated in cognitive tasks like directing visually guided actions. In order to better understand the underlying circuitry, we performed retrograde injections of modified rabies virus in the visual cortex and pulvinar of the Long-Evans rat. We found a relatively small population of cells projecting to primary visual cortex (V1), compared to a much larger population projecting to higher visual cortex. Reciprocal corticothalamic projections showed a similar result, implying that pulvinar does not play as big a role in directly modulating rodent V1 activity as previously thought.

Sex related biases for attending to object color versus object position are reflected in reaction time and accuracy

Processing of visual features related to objects and space relations occurs within separate cortical streams that interact with selective attention. Such separation has implications for cognitive development because the perception of ‘what’ and ‘where’ provide a neural foundation for the development of aspects of higher cognition. Thus, a small attentional bias in early development for attending to one aspect over the other might influence subsequent higher cognitive processing in tasks involving object recognition and space relations. We examined 134 men and women for evidence of an inherent sex-related bias for attending to basic perceptual features related to object discrimination versus object position. Each stimulus consisted of a circle located in one of 9 positions within a surrounding frame. Circles were one of three shades of blue or red. These stimuli were used in a match-to-sample paradigm where participants were required to match circles on the basis of color or spatial position. The first stimulus appeared in the center of the screen for 400 msec and the matching stimulus subsequently appeared for 400 msec oriented 5 degrees to the right or left of center. The same stimuli were used to test the perception of color and position, with order of testing counterbalanced across participants. Results showed significantly longer reaction times in females compared with males, with better accuracy to discriminate color when that color was tested before position. Males showed better accuracy when object position was tested before color discrimination. A second experiment employed the same procedure, but enhanced selective attention by adding an endogenous cue that predicted the right or left location for the appearance of the matching stimulus. This manipulation greatly attenuated the sex differences in reaction time and accuracy compared to Experiment 1, suggesting that the sex-related attentional biases are strongly coupled to bottom-up processing. Overall, the sex related attentional biases toward processing object characteristics versus object position location suggest a differential manifestation of biased competition between the weighted systems of dorsal and ventral stream processing. Results are discussed with how a developmental bias in the processing objects versus space relations may contribute to adult cognitive sex differences in humans and animals.

Eccentricity-dependent temporal contrast tuning in human visual cortex measured with fMRI

Cells in the peripheral retina tend to have higher contrast sensitivity and respond at higher flicker frequencies than those closer to the fovea. Although this predicts increased behavioural temporal contrast sensitivity in the peripheral visual field, this effect is rarely observed in psychophysical experiments. It is unknown how temporal contrast sensitivity is represented across eccentricity within cortical visual field maps and whether such sensitivities reflect the response properties of retinal cells or psychophysical sensitivities. Here, we used functional magnetic resonance imaging (fMRI) to measure contrast sensitivity profiles at four temporal frequencies in five retinotopically-defined visual areas. We also measured population receptive field (pRF) parameters (polar angle, eccentricity, and size) in the same areas. Overall contrast sensitivity, independent of pRF parameters, peaked at 10 Hz in all visual areas. In V1, V2, V3, and V3a, peripherally-tuned voxels had higher contrast sensitivity at a high temporal frequency (20 Hz), while hV4 more closely reflected behavioural sensitivity profiles. We conclude that our data reflect a cortical representation of the increased peripheral temporal contrast sensitivity that is already present in the retina and that this bias must be compensated later in the cortical visual pathway.

Graph measures in task-based fMRI: Functional integration during read-out of visual and auditory information

This study investigated how attending to auditory and visual information systematically changes graph theoretical measures of integration and functional connectivity between three network modules: auditory, visual, and a joint task core. Functional MRI BOLD activity was recorded while healthy volunteers attended to colour and/or pitch information presented within an audiovisual stimulus sequence. Network nodes and modules were based on peak voxels of BOLD contrasts, including colour and pitch sensitive brain regions as well as the dorsal attention network. Network edges represented correlations between nodes' activity and were computed separately for each condition. Connection strength was increased between the task and the visual module when participants attended to colour, and between the task and the auditory module when they attended to pitch. Moreover, several nodal graph measures showed consistent changes to attentional modulation in form of stronger integration of sensory regions in response to attention. Together, these findings corroborate dynamical adjustments of both modality-specific and modality-independent functional brain networks in response to task demands and their representation in graph theoretical measures.

Neuronal correlates of motion-defined shape perception in primate dorsal and ventral streams

Human and non-human primates can readily perceive the shape of objects using visual motion. Classically, shape, and motion are considered to be separately processed via ventral and dorsal cortical pathways, respectively. However, many lines of anatomical and physiological evidence have indicated that these two pathways are likely to be interconnected at some stage. For motion-defined shape perception, these two pathways should interact with each other because the ventral pathway must utilize motion, which the dorsal pathway processes, to extract shape signal. However, it is unknown how interactions between cortical pathways are involved in neural mechanisms underlying motion-defined shape perception. We review evidence from psychophysical, lesion, neuroimaging and physiological research on motion-defined shape perception and then discuss the effects of behavioral demands on neural activity in ventral and dorsal cortical areas. Further, we discuss functions of two candidate sets of levels: early and higher-order cortical areas. The extrastriate area V4 and middle temporal (MT) area, which are reciprocally connected, at the early level are plausible areas for extracting the shape and/or constituent parts of shape from motion cues because neural dynamics are different from those during luminance-defined shape perception. On the other hand, among other higher-order visual areas, the anterior superior temporal sulcus likely contributes to the processing of cue-invariant shape recognition rather than cue-dependent shape processing. We suggest that sharing information about motion and shape between the early visual areas in the dorsal and ventral pathways is dependent on visual cues and behavioral requirements, indicating the interplay between the pathways.

Towards a unified perspective of object shape and motion processing in human dorsal cortex

Although object-related areas were discovered in human parietal cortex a decade ago, surprisingly little is known about the nature and purpose of these representations, and how they differ from those in the ventral processing stream. In this article, we review evidence for the unique contribution of object areas of dorsal cortex to three-dimensional (3-D) shape representation, the localization of objects in space, and in guiding reaching and grasping actions. We also highlight the role of dorsal cortex in form-motion interaction and spatiotemporal integration, possible functional relationships between 3-D shape and motion processing, and how these processes operate together in the service of supporting goal-directed actions with objects. Fundamental differences between the nature of object representations in the dorsal versus ventral processing streams are considered, with an emphasis on how and why dorsal cortex supports veridical (rather than invariant) representations of objects to guide goal-directed hand actions in dynamic visual environments.

Novel method of extracting motion from natural movies

BACKGROUND

The visual system in primates can be segregated into motion and shape pathways. Interaction occurs at multiple stages along these pathways. Processing of shape-from-motion and biological motion is considered to be a higher-order integration process involving motion and shape information. However, relatively limited types of stimuli have been used in previous studies on these integration processes.

NEW METHOD

We propose a new algorithm to extract object motion information from natural movies and to move random dots in accordance with the information. The object motion information is extracted by estimating the dynamics of local normal vectors of the image intensity projected onto the x-y plane of the movie.

RESULTS

An electrophysiological experiment on two adult common marmoset monkeys (Callithrix jacchus) showed that the natural and random dot movies generated with this new algorithm yielded comparable neural responses in the middle temporal visual area.

COMPARISON WITH EXISTING METHODS

In principle, this algorithm provided random dot motion stimuli containing shape information for arbitrary natural movies. This new method is expected to expand the neurophysiological and psychophysical experimental protocols to elucidate the integration processing of motion and shape information in biological systems.

CONCLUSIONS

The novel algorithm proposed here was effective in extracting object motion information from natural movies and provided new motion stimuli to investigate higher-order motion information processing.

Neural representation for object recognition in inferotemporal cortex

We suggest that population representation of objects in inferotemporal cortex lie on a continuum between a purely structural, parts-based description and a purely holistic description. The intrinsic dimensionality of object representation is estimated to be around 100, perhaps with lower dimensionalities for object representations more toward the holistic end of the spectrum. Cognitive knowledge in the form of semantic information and task information feed back to inferotemporal cortex from perirhinal and prefrontal cortex respectively, providing high-level multimodal-based expectations that assist in the interpretation of object stimuli. Integration of object information across eye movements may also contribute to object recognition through a process of active vision.

Neural pathways for visual speech perception

This paper examines the questions, what levels of speech can be perceived visually, and how is visual speech represented by the brain? Review of the literature leads to the conclusions that every level of psycholinguistic speech structure (i.e., phonetic features, phonemes, syllables, words, and prosody) can be perceived visually, although individuals differ in their abilities to do so; and that there are visual modality-specific representations of speech qua speech in higher-level vision brain areas. That is, the visual system represents the modal patterns of visual speech. The suggestion that the auditory speech pathway receives and represents visual speech is examined in light of neuroimaging evidence on the auditory speech pathways. We outline the generally agreed-upon organization of the visual ventral and dorsal pathways and examine several types of visual processing that might be related to speech through those pathways, specifically, face and body, orthography, and sign language processing. In this context, we examine the visual speech processing literature, which reveals widespread diverse patterns of activity in posterior temporal cortices in response to visual speech stimuli. We outline a model of the visual and auditory speech pathways and make several suggestions: (1) The visual perception of speech relies on visual pathway representations of speech qua speech. (2) A proposed site of these representations, the temporal visual speech area (TVSA) has been demonstrated in posterior temporal cortex, ventral and posterior to multisensory posterior superior temporal sulcus (pSTS). (3) Given that visual speech has dynamic and configural features, its representations in feedforward visual pathways are expected to integrate these features, possibly in TVSA.