Representation of perceived object shape by the human lateral occipital complex

Communication of perceptual predictions from the hippocampus to the deep layers of the parahippocampal cortex

Current evidence suggests that the hippocampus is essential for exploiting predictive relationships during perception. However, it remains unclear whether the hippocampus drives the communication of predictions to sensory cortex or receives prediction signals from elsewhere. We collected 7-tesla fMRI data in the medial temporal lobe (MTL) while auditory cues predicted abstract shapes. Strikingly, neural patterns evoked by predicted shapes in CA2/3, pre/parasubiculum, and the parahippocampal cortex (PHC) were negatively correlated to patterns evoked by the same shapes when actually presented. Using layer-specific analyses, we ask: In which direction are predictions communicated between the hippocampus and neocortex? Superficial layers of the MTL cortex project to the hippocampus, while the deep layers receive feedback projections. Informational connectivity analyses revealed that communication between CA2/3 and PHC was specific to the deep layers of PHC. These findings suggest that the hippocampus generates predictions through pattern completion in CA2/3 and feeds these predictions back to the neocortex.

Neural mechanisms of symmetry perception: hemispheric specialization and the impact of noise on reflection symmetry detection

Symmetry is a crucial cue for perceptual grouping in human vision. This study investigates the neural and cognitive mechanisms underlying symmetry perception, focusing on hemispheric specialization and the effects of noise on symmetry detection. Using psychophysical and electrophysiological (EEG) experiments, participants were presented with reflection symmetric patterns (full circle vs. right-left quarter-circle), under varying noise levels. Behavioral results demonstrated noise-induced impairment in accuracy (p < 0.001), with Cycle outperforming Quarter in noiseless conditions (p < 0.05), highlighting the role of contour completeness in perceptual grouping. EEG recordings revealed distinct neural mechanisms associated with different stages of symmetry processing. Early sensory processing exhibited left-hemisphere dominance, while later stages implicated the right hemisphere in noise-modulated global integration. Noise disrupted early contour integration and attenuated higher-order object recognition processes, with right-hemisphere sensitivity to noise emerging during decision-making. These findings challenge the strong version of the callosal hypothesis, highlighting the complexity of hemispheric interactions in symmetry perception. This study provides new insights into the interplay between bottom-up sensory processing and top-down hemispheric interactions in perceptual organization.

Functional perspectives in mental jigsaw puzzles: Insights from eye-tracking, questionnaire, and behavioral data

This study investigated cognitive strategies in mental jigsaw puzzles, integrating mental rotation and translation with a focus on directionality and detour arguments. Unlike object mental rotation tasks, these puzzles introduced physical constraints, revealing systematic directional tendencies in both eye movements and subjective reports. Specifically, smaller protruding objects were consistently directed toward larger indented objects. This was accompanied by longer completion times and reduced linearity, paralleling strategies used in physical puzzle-solving. Behavioral asymmetries observed in the puzzles unexpectedly mirrored those found in object mental rotation tasks. While controlling for mental motion directions showed comparable completion times at 300° between tasks, the study did not fully clarify the role of detours, indicating the need for further research.

Visual mental imagery in typical imagers and in aphantasia: A millimeter-scale 7-T fMRI study

Most of us effortlessly describe visual objects, whether seen or remembered. Yet, around 4% of people report congenital aphantasia: they struggle to visualize objects despite being able to describe their visual appearance. What neural mechanisms create this disparity between subjective experience and objective performance? Aphantasia can provide novel insights into conscious processing and awareness. We used ultra-high field 7T fMRI to establish the neural circuits involved in visual mental imagery and perception, and to elucidate the neural mechanisms associated with the processing of internally generated visual information in the absence of imagery experience in congenital aphantasia. Ten typical imagers and 10 aphantasic individuals performed imagery and perceptual tasks in five domains: object shape, object color, written words, faces, and spatial relationships. In typical imagers, imagery tasks activated left-hemisphere frontoparietal areas, the relevant domain-preferring areas in the ventral temporal cortex partly overlapping with the perceptual domain-preferring areas, and a domain-general area in the left fusiform gyrus (the Fusiform Imagery Node). The results were valid for each individual participant. In aphantasic individuals, imagery activated similar visual areas, but there was reduced functional connectivity between the Fusiform Imagery Node and frontoparietal areas. Our results unveil the domain-general and domain-specific circuits of visual mental imagery, their functional disorganization in aphantasia, and support the general hypothesis that conscious visual experience - whether perceived or imagined - depends on the integrated activity of high-level visual cortex and frontoparietal networks.

Visual categorisation of images of familiar objects based on their authenticity: an fMRI study

Under most circumstances, we can rely visual information to quickly and accurately discriminate "real" objects (e.g., fresh fruit) from "fake" objects (e.g., plastic fruit). It is unclear, however, whether this distinction is made early along the ventral visual stream when basic object features such as colour (e.g., primary visual cortex; V1) and texture (e.g., collateral sulcus; COS) are being processed, or whether information regarding object authenticity is extracted in later visual or memory regions (e.g., perirhinal cortex, lateral occipital cortex). To examine this question, participants were placed in an fMRI scanner, and presented with 300 objects photographed in colour or greyscale. Half of the objects were fake, and the other half were real. The participant's task was to categorise each image as presenting either a real or fake object. Broadly, our analyses revealed significant activation in CoS when participants categorised real objects, particularly when they were presented in colour. We also observed activation in V1 for coloured objects, particularly real ones. These results suggest that our seemingly intuitive ability to rapidly discriminate real from fake objects occurs at the early stages of visual processing, such as when the brain is extracting surface-feature information like texture (CoS) or colour (V1). Future studies could consider the time course of these neural events and probe the importance of cross-modal (e.g., audition and haptic) information underpinning feature extraction for distinguishing real from fake objects.

Neurocognitive dynamics and behavioral differences of symmetry and asymmetry processing in working memory: insights from fNIRS

Symmetry is a ubiquitous property of the visual world. It facilitates cognitive processing and fosters aesthetic appeal. Despite its importance to aesthetic experience and perceptual prominence, the integration of symmetry in working memory remains underexplored. In our study, participants engaged in a novel working memory task involving both symmetrical and asymmetrical stimuli, while their brain activity was monitored using functional Near Infrared Spectroscopy (fNIRS). The study revealed that symmetry significantly enhances memory performance. Symmetry significantly improves task performance, with symmetrical stimuli leading to higher accuracy and faster recall than asymmetrical ones, especially under high cognitive load. This effect varies with the type of symmetry, with diagonal symmetry being the most effective. Neuroimaging data showed distinct brain activation patterns when participants processed symmetrical stimuli, particularly in the memory-straining condition. Significant differences in brain activity were observed in various brain regions, with lateral occipital, posterior parietal, medial and dorsolateral prefrontal cortices reacting to symmetry with decreased oxygenated hemoglobin (HbO), while in left orbitofrontal (HbO) and right ventrolateral prefrontal cortex (HbO and HbR) hemoglobin concentration increased. Overall, our findings highlight the complex, region-specific brain activation patterns in response to visual symmetry, emphasizing the nuanced role of symmetry in cognitive processing during memory tasks and their potential implication for creative thinking.

Disentangling the neural underpinnings of response inhibition in disruptive behavior and co-occurring ADHD

While impaired response inhibition has been reported in attention-deficit/hyperactivity disorder (ADHD), findings in disruptive behavior disorders (DBDs) have been inconsistent, probably due to unaccounted effects of co-occurring ADHD in DBD. This study investigated the associations of behavioral and neural correlates of response inhibition with DBD and ADHD symptom severity, covarying for each other in a dimensional approach. Functional magnetic resonance imaging data were available for 35 children and adolescents with DBDs (8-18 years old, 19 males), and 31 age-matched unaffected controls (18 males) while performing a performance-adjusted stop-signal task. No significant association was found between behavioral performance and symptom severities. However, contrasting successful inhibition with failed inhibition revealed that DBD and ADHD symptom severity was associated with greater activation in the right inferior frontal regions and reduced activation in the bilateral striatal regions, respectively. During successful inhibition versus go-trials, ADHD symptom severity was associated with the left lateral occipital cortex activation. The contrast of failed inhibition versus go-trials revealed reduced activation in the right frontal and left parietal regions associated with DBD symptom severity while ADHD symptom severity was associated with bilateral precunei, dorsolateral prefrontal and left posterior parietal regions. Except for the right inferior frontal regions during successful versus failed inhibition, all clusters were also found to be inversely associated with the other dimension of interest (i.e., DBD or ADHD symptoms). Opposite direction of the associations between DBD and ADHD symptom severity, and fronto-parietal and fronto-striatal activation suggest unique contributions of DBD and ADHD to the neural correlates of response inhibition.

Towards reliable object representation via sparse directional patches and spatial center cues

In the process of image understanding, the human visual system (HVS) performs multiscale analysis on various objects. HVS primarily focuses on marginally conspicuous image patches located within or around distinct objects rather than scanning the image pixels point by point. Inspired by the HVS mechanism, in this paper, we aimed to describe and exploit multiscale decomposition-based patch detection models for automatic visual feature representation and object localization in images. Our investigation into mimicking and modeling the HVS to capture conspicuous sparse patches and their spatial distribution clues makes a profound contribution to the automatic comprehension and characterization of images by machines. This study demonstrates that the sparse patch-based visual representation with spatial center cues is intrinsically tolerant to object positioning and understanding beyond object variations in spatial position, multiresolution, and chrominance, which has significant implications for many vision-based automatic object grabbing and perception applications, such as robotics, human‒machine interaction, and unmanned aerial vehicles (UAVs).

Seeing and visualizing across the hemispheres

Despite our subjective experience of a largely symmetric visual world, the human brain exhibits varying patterns and degrees of hemispheric asymmetry in distinct processes of visual cognition. This chapter reviews behavioral and neuroimaging evidence from neurotypical individuals and neurological patients, concerning functional asymmetries between the right hemisphere (RH) and the left hemisphere (LH) in visual object processing and mental imagery. Hierarchical perception shows RH preference for global processing and LH preference for local processing. At later stages of visual object processing, RH-based circuits exhibit a relative advantage in terms of perceptual integration, with a subsequent shift toward LH-based circuits for processing at higher conceptual and semantic levels. In voluntary visual mental imagery, circuits in the LH ventral temporal cortex play a pivotal role in transitioning from object meaning to simulated visualization. These hemispheric asymmetries in visual object processing might, in part, be influenced by the overall need to minimize wiring, coupled with the presence of distinct specialized networks within each hemisphere, such as the RH attention networks and the LH language networks. From a broader viewpoint, the evidence examined in this chapter indicates that visual object processing involves the interactions of large-scale cortical circuits within and between the hemispheres.

Intracortical recordings reveal the neuronal selectivity for bodies and body parts in the human visual cortex

Body perception plays a fundamental role in social cognition. Yet, the neural mechanisms underlying this process in humans remain elusive given the spatiotemporal constraints of functional imaging. Here, we present intracortical recordings of single- and multiunit spiking activity in two epilepsy surgery patients in or near the extrastriate body area, a critical region for body perception. Our recordings revealed a strong preference for human bodies over a large range of control stimuli. Notably, body selectivity was driven by a distinct selectivity for body parts. The observed body selectivity generalized to nonphotographic depictions of bodies including silhouettes and stick figures. Overall, our study provides unique neural data that bridge the gap between human neuroimaging and macaque electrophysiology studies, laying a solid foundation for computational models of human body processing.

Glutamate dynamics and BOLD response during OCD symptom provocation in the lateral occipital cortex: A 7 Tesla fMRI-fMRS study

Obsessive-compulsive disorder (OCD) is linked with dysfunction in frontal-striatal, fronto-limbic, and visual brain regions. Research using proton magnetic resonance spectroscopy (1H-MRS) suggests that altered neurometabolite levels, like glutamate, may contribute to this dysfunction. However, static neurometabolite levels in OCD patients have shown inconsistent results, likely due to previous studies' limited focus on neurometabolite dynamics. We employ functional MRS (fMRS) and functional magnetic resonance imaging (fMRI) to explore these dynamics and brain activation during OCD symptom provocation. We utilized a combined 7-tesla fMRI-fMRS setup to examine task-related BOLD response and glutamate changes in the lateral occipital cortex (LOC) of 30 OCD participants and 34 matched controls during an OCD-specific symptom provocation task. The study examined main effects and between-group differences in brain activation and glutamate levels during the task. A whole sample task-effects analysis on data meeting predefined quality criteria showed significant glutamate increases (n = 41 (22 OCD, 19 controls), mean change: 3.2 %, z = 3.75, p < .001) and task activation (n = 54 (26 OCD, 28 controls), p < .001) in the LOC during OCD blocks compared to neutral blocks. However, no differences in task-induced glutamate dynamics or activation between groups were found, nor a correlation between glutamate levels and task activation. We were able to measure task-induced increases in glutamate and BOLD levels, emphasizing its feasibility for OCD research. The absence of group differences highlights the need for further exploration to discern to what extent neurometabolite dynamics differ between OCD patients and controls. Once established, future studies can use pre-post intervention fMRS-fMRI to probe the effects of therapies modulating glutamate pathways in OCD.

Using imagination and the contents of memory to create new scene and object representations: A functional MRI study

Humans can use the contents of memory to construct scenarios and events that they have not encountered before, a process colloquially known as imagination. Much of our current understanding of the neural mechanisms mediating imagination is limited by paradigms that rely on participants' subjective reports of imagined content. Here, we used a novel behavioral paradigm that was designed to systematically evaluate the contents of an individual's imagination. Participants first learned the layout of four distinct rooms containing five wall segments with differing geometrical characteristics, each associated with a unique object. During functional MRI, participants were then shown two different wall segments or objects on each trial and asked to first, retrieve the associated objects or walls, respectively (retrieval phase) and then second, imagine the two objects side-by-side or combine the two wall segments (imagination phase). Importantly, the contents of each participant's imagination were interrogated by having them make a same/different judgment about the properties of the imagined objects or scenes. Using univariate and multivariate analyses, we observed widespread activity across occipito-temporal cortex for the retrieval of objects and for the imaginative creation of scenes. Interestingly, a classifier, whether trained on the imagination or retrieval data, was able to successfully differentiate the neural patterns associated with the imagination of scenes from that of objects. Our results reveal neural differences in the cued retrieval of object and scene memoranda, demonstrate that different representations underlie the creation and/or imagination of scene and object content, and highlight a novel behavioral paradigm that can be used to systematically evaluate the contents of an individual's imagination.

High-level visual prediction errors in early visual cortex

Perception is shaped by both incoming sensory input and expectations derived from our prior knowledge. Numerous studies have shown stronger neural activity for surprising inputs, suggestive of predictive processing. However, it is largely unclear what predictions are made across the cortical hierarchy, and therefore what kind of surprise drives this up-regulation of activity. Here, we leveraged fMRI in human volunteers and deep neural network (DNN) models to arbitrate between 2 hypotheses: prediction errors may signal a local mismatch between input and expectation at each level of the cortical hierarchy, or prediction errors may be computed at higher levels and the resulting surprise signal is broadcast to earlier areas in the cortical hierarchy. Our results align with the latter hypothesis. Prediction errors in both low- and high-level visual cortex responded to high-level, but not low-level, visual surprise. This scaling with high-level surprise in early visual cortex strongly diverged from feedforward tuning. Combined, our results suggest that high-level predictions constrain sensory processing in earlier areas, thereby aiding perceptual inference.

Dependence of Contextual Modulation in Macaque V1 on Interlaminar Signal Flow

In visual cortex, neural correlates of subjective perception can be generated by modulation of activity from beyond the classical receptive field (CRF). In macaque V1, activity generated by nonclassical receptive field (nCRF) stimulation involves different intracortical circuitry than activity generated by CRF stimulation, suggesting that interactions between neurons across V1 layers differ under CRF and nCRF stimulus conditions. Using Neuropixels probes, we measured border ownership modulation within large, local populations of V1 neurons. We found that neurons in single columns preferred the same side of objects located outside of the CRF. In addition, we found that cross-correlations between pairs of neurons situated across feedback/horizontal and input layers differed between CRF and nCRF stimulation. Furthermore, independent of the comparison with CRF stimulation, we observed that the magnitude of border ownership modulation increased with the proportion of information flow from feedback/horizontal layers to input layers. These results demonstrate that the flow of signals between layers covaries with the degree to which neurons integrate information from beyond the CRF.

Distributed network flows generate localized category selectivity in human visual cortex

A central goal of neuroscience is to understand how function-relevant brain activations are generated. Here we test the hypothesis that function-relevant brain activations are generated primarily by distributed network flows. We focused on visual processing in human cortex, given the long-standing literature supporting the functional relevance of brain activations in visual cortex regions exhibiting visual category selectivity. We began by using fMRI data from N = 352 human participants to identify category-specific responses in visual cortex for images of faces, places, body parts, and tools. We then systematically tested the hypothesis that distributed network flows can generate these localized visual category selective responses. This was accomplished using a recently developed approach for simulating - in a highly empirically constrained manner - the generation of task-evoked brain activations by modeling activity flowing over intrinsic brain connections. We next tested refinements to our hypothesis, focusing on how stimulus-driven network interactions initialized in V1 generate downstream visual category selectivity. We found evidence that network flows directly from V1 were sufficient for generating visual category selectivity, but that additional, globally distributed (whole-cortex) network flows increased category selectivity further. Using null network architectures we also found that each region's unique intrinsic "connectivity fingerprint" was key to the generation of category selectivity. These results generalized across regions associated with all four visual categories tested (bodies, faces, places, and tools), and provide evidence that the human brain's intrinsic network organization plays a prominent role in the generation of functionally relevant, localized responses.

The "What" and "How" of Pantomime Actions

Pantomimes are human actions that simulate ideas, objects, and events, commonly used in conversation, performance art, and gesture-based interfaces for computing and controlling robots. Yet, their underlying neurocognitive mechanisms are not well understood. In this review, we examine pantomimes through two parallel lines of research: (1) the two visual systems (TVS) framework for visually guided action, and (2) the neuropsychological literature on limb apraxia. Historically, the TVS framework has considered pantomime actions as expressions of conscious perceptual processing in the ventral stream, but an emerging view is that they are jointly influenced by ventral and dorsal stream processing. Within the apraxia literature, pantomimes were historically viewed as learned motor schemas, but there is growing recognition that they include creative and improvised actions. Both literatures now recognize that pantomimes are often created spontaneously, sometimes drawing on memory and always requiring online cognitive control. By highlighting this convergence of ideas, we aim to encourage greater collaboration across these two research areas, in an effort to better understand these uniquely human behaviors.

Stimulus repetition induces a two-stage learning process in primary visual cortex

Repeated stimulus exposure alters the brain's response to the stimulus. We investigated the underlying neural mechanisms by recording functional MRI data from human observers passively viewing 120 presentations of two Gabor patches (each Gabor repeating 60 times). We evaluated support for two prominent models of stimulus repetition, the fatigue model and the sharpening model. Our results uncovered a two-stage learning process in the primary visual cortex. In Stage 1, univariate BOLD activation in V1 decreased over the first twelve repetitions of the stimuli, replicating the well-known effect of repetition suppression. Applying MVPA decoding along with a moving window approach, we found that (1) the decoding accuracy between the two Gabors decreased from above-chance level (∼60% to ∼70%) at the beginning of the stage to chance level at the end of the stage (∼50%). This result, together with the accompanying weight map analysis, suggested that the learning dynamics in Stage 1 were consistent with the predictions of the fatigue model. In Stage 2, univariate BOLD activation for the remaining 48 repetitions of the two stimuli exhibited significant fluctuations but no systematic trend. The MVPA decoding accuracy between the two Gabor patches was at chance level initially and became progressively higher as stimulus repetition continued, rising above and staying above chance level starting at the ∼35th repetition. Thus, results from the second stage supported the notion that sustained and prolonged stimulus repetition prompts sharpened representations. Additional analyses addressed (1) whether the neural patterns within each learning stage remained stable and (2) whether new neural patterns were evoked in Stage 2 relative to Stage 1.

Using Illusions to Track the Emergence of Visual Perception

Everybody loves illusions. At times, the content on the internet seems to be mostly about illusions-shoes, dresses, straight lines looking bent. This attraction has a long history. Almost 2,000 years ago, Ptolemy marveled at how the sail of a distant boat could appear convex or concave. This sense of marvel continues to drive our fascination with illusions; indeed, few other corners of science can boast of such a large reach. However, illusions not only draw in the crowds; they also offer insights into visual processes. This review starts with a simple definition of illusions as conflicts between perception and cognition, where what we see does not agree with what we believe we should see. This mismatch can be either because cognition has misunderstood how perception works or because perception has misjudged the visual input. It is the perceptual errors that offer the chance to track the development of perception across visual regions. Unfortunately, the effects of illusions in different brain regions cannot be isolated in any simple way: Top-down projections from attention broadcast the expected perceptual properties everywhere, obscuring the critical evidence of where the illusion and perception emerge. The second part of this review then highlights the roadblocks to research raised by attention and describes current solutions for accessing what illusions can offer.

The role of task on the human brain's responses to, and representation of, visual regularity defined by reflection and rotation

Identifying and segmenting objects in an image is generally achieved effortlessly and is facilitated by the presence of symmetry: a principle of perceptual organisation used to interpret sensory inputs from the retina into meaningful representations. However, while imaging studies show evidence of symmetry selective responses across extrastriate visual areas in the human brain, whether symmetry is processed automatically is still under debate. We used functional Magnetic Resonance Imaging (fMRI) to study the response to and representation of two types of symmetry: reflection and rotation. Dot pattern stimuli were presented to 15 human participants (10 female) under stimulus-relevant (symmetry) and stimulus-irrelevant (luminance) task conditions. Our results show that symmetry-selective responses emerge from area V3 and extend throughout extrastriate visual areas. This response is largely maintained when participants engage in the stimulus irrelevant task, suggesting an automaticity to processing visual symmetry. Our multi-voxel pattern analysis (MVPA) results extend these findings by suggesting that not only spatial organisation of responses to symmetrical patterns can be distinguished from that of non-symmetrical (random) patterns, but also that representation of reflection and rotation symmetry can be differentiated in extrastriate and object-selective visual areas. Moreover, task demands did not affect the neural representation of the symmetry information. Intriguingly, our MVPA results show an interesting dissociation: representation of luminance (stimulus irrelevant feature) is maintained in visual cortex only when task relevant, while information of the spatial configuration of the stimuli is available across task conditions. This speaks in favour of the automaticity for processing perceptual organisation: extrastriate visual areas compute and represent global, spatial properties irrespective of the task at hand.

Object-based attention requires monocular visual pathways

Mechanisms of object-based attention (OBA) are commonly associated with the cerebral cortex. However, less is known about the involvement of subcortical visual pathways in these processes. Knowledge of the neural mechanisms subserving OBA can provide insight into the evolutionary trajectory of attentional selection. In the current study, the classic double-rectangle cueing task was implemented using a stereoscope in order to differentiate between the involvement of lower (monocular) and higher (binocular) visual pathways in OBA processes. We found that monocular visual pathways are involved in two main aspects of OBA: exogenous orienting towards a cued object (Experiment 1; N =33) and attentional deployment within a cued object (Experiment 2; N =23); this is evident by the presence of OBA only when both the cue and target were presented to the same eye. Thus, these results indicate that monocular (mostly subcortical) visual regions are not simply passing information to higher cortical areas but have a functional computational role in OBA. These findings emphasize the importance of lower regions in attentional processes and, more specifically, in OBA.

Preserved recognition of basic visual features despite lack of awareness of shape: Evidence from a case of neglect

Human visual experience of objects comprises a combination of visual features, such as color, position, and shape. Spatial attention is thought to play a role in creating a coherent perceptual experience, integrating visual information coming from a given location, but the mechanisms underlying this process are not fully understood. Deficits of spatial attention in which this integration process does not occur normally, such as neglect, can provide insights regarding the mechanisms of spatial attention in visual object recognition. In this study, we describe a series of experiments conducted with an individual with neglect, DH. DH presents characteristic lack of awareness of the left side of individual objects, evidenced by poor object and face recognition, and impaired word reading. However, he exhibits intact recognition of color within the boundaries of the same objects he fails to recognize. Furthermore, he can also report the orientation and location of a colored region on the neglected left side despite lack of awareness of the shape of the region. Overall, DH shows selective lack of awareness of shape despite intact processing of basic visual features in the same spatial location. DH's performance raises intriguing questions and challenges about the role of spatial attention in the formation of coherent object percepts and visual awareness.

Influence of gaze, vision, and memory on hand kinematics in a placement task

People usually reach for objects to place them in some position and orientation, but the placement component of this sequence is often ignored. For example, reaches are influenced by gaze position, visual feedback, and memory delays, but their influence on object placement is unclear. Here, we tested these factors in a task where participants placed and oriented a trapezoidal block against two-dimensional (2-D) visual templates displayed on a frontally located computer screen. In experiment 1, participants matched the block to three possible orientations: 0° (horizontal), +45° and -45°, with gaze fixated 10° to the left/right. The hand and template either remained illuminated (closed-loop), or visual feedback was removed (open-loop). Here, hand location consistently overshot the template relative to gaze, especially in the open-loop task; likewise, orientation was influenced by gaze position (depending on template orientation and visual feedback). In experiment 2, a memory delay was added, and participants sometimes performed saccades (toward, away from, or across the template). In this task, the influence of gaze on orientation vanished, but location errors were influenced by both template orientation and final gaze position. Contrary to our expectations, the previous saccade metrics also impacted placement overshoot. Overall, hand orientation was influenced by template orientation in a nonlinear fashion. These results demonstrate interactions between gaze and orientation signals in the planning and execution of hand placement and suggest different neural mechanisms for closed-loop, open-loop, and memory delay placement.NEW & NOTEWORTHY Eye-hand coordination studies usually focus on object acquisition, but placement is equally important. We investigated how gaze position influences object placement toward a 2-D template with different levels of visual feedback. Like reach, placement overestimated goal location relative to gaze and was influenced by previous saccade metrics. Gaze also modulated hand orientation, depending on template orientation and level of visual feedback. Gaze influence was feedback-dependent, with location errors having no significant effect after a memory delay.

The large-scale organization of shape processing in the ventral and dorsal pathways is dissociable from attention

The two visual pathways model posits that visual information is processed through two distinct cortical systems: The ventral pathway promotes visual recognition, while the dorsal pathway supports visuomotor control. Recent evidence suggests the dorsal pathway is also involved in shape processing and may contribute to object perception, but it remains unclear whether this sensitivity is independent of attentional mechanisms that were localized to overlapping cortical regions. To address this question, we conducted two fMRI experiments that utilized different parametric scrambling manipulations in which human participants viewed novel objects in different levels of scrambling and were instructed to attend to either the object or to another aspect of the image (e.g. color of the background). Univariate and multivariate analyses revealed that the large-scale organization of shape selectivity along the dorsal and ventral pathways was preserved regardless of the focus of attention. Attention did modulate shape sensitivity, but these effects were similar across the two pathways. These findings support the idea that shape processing is at least partially dissociable from attentional processes and relies on a distributed set of cortical regions across the visual pathways.

Amodal completion across the brain: The impact of structure and knowledge

This study investigates the phenomenon of amodal completion within the context of naturalistic objects, employing a repetition suppression paradigm to disentangle the influence of structure and knowledge cues on how objects are completed. The research focuses on early visual cortex (EVC) and lateral occipital complex (LOC), shedding light on how these brain regions respond to different completion scenarios. In LOC, we observed suppressed responses to structure and knowledge-compatible stimuli, providing evidence that both cues influence neural processing in higher-level visual areas. However, in EVC, we did not find evidence for differential responses to completions compatible or incompatible with either structural or knowledge-based expectations. Together, our findings suggest that the interplay between structure and knowledge cues in amodal completion predominantly impacts higher-level visual processing, with less pronounced effects on the early visual cortex. This study contributes to our understanding of the complex mechanisms underlying visual perception and highlights the distinct roles played by different brain regions in amodal completion.

Neuronal tuning and population representations of shape and category in human visual cortex

Object recognition and categorization are essential cognitive processes which engage considerable neural resources in the human ventral visual stream. However, the tuning properties of human ventral stream neurons for object shape and category are virtually unknown. We performed large-scale recordings of spiking activity in human Lateral Occipital Complex in response to stimuli in which the shape dimension was dissociated from the category dimension. Consistent with studies in nonhuman primates, the neuronal representations were primarily shape-based, although we also observed category-like encoding for images of animals. Surprisingly, linear decoders could reliably classify stimulus category even in data sets that were entirely shape-based. In addition, many recording sites showed an interaction between shape and category tuning. These results represent a detailed study on shape and category coding at the neuronal level in the human ventral visual stream, furnishing essential evidence that reconciles human imaging and macaque single-cell studies.

Naturalistic Object Representations Depend on Distance and Size Cues

Egocentric distance and real-world size are important cues for object perception and action. Nevertheless, most studies of human vision rely on two-dimensional pictorial stimuli that convey ambiguous distance and size information. Here, we use fMRI to test whether pictures are represented differently in the human brain from real, tangible objects that convey unambiguous distance and size cues. Participants directly viewed stimuli in two display formats (real objects and matched printed pictures of those objects) presented at different egocentric distances (near and far). We measured the effects of format and distance on fMRI response amplitudes and response patterns. We found that fMRI response amplitudes in the lateral occipital and posterior parietal cortices were stronger overall for real objects than for pictures. In these areas and many others, including regions involved in action guidance, responses to real objects were stronger for near vs. far stimuli, whereas distance had little effect on responses to pictures-suggesting that distance determines relevance to action for real objects, but not for pictures. Although stimulus distance especially influenced response patterns in dorsal areas that operate in the service of visually guided action, distance also modulated representations in ventral cortex, where object responses are thought to remain invariant across contextual changes. We observed object size representations for both stimulus formats in ventral cortex but predominantly only for real objects in dorsal cortex. Together, these results demonstrate that whether brain responses reflect physical object characteristics depends on whether the experimental stimuli convey unambiguous information about those characteristics.

Significance Statement

Classic frameworks of vision attribute perception of inherent object characteristics, such as size, to the ventral visual pathway, and processing of spatial characteristics relevant to action, such as distance, to the dorsal visual pathway. However, these frameworks are based on studies that used projected images of objects whose actual size and distance from the observer were ambiguous. Here, we find that when object size and distance information in the stimulus is less ambiguous, these characteristics are widely represented in both visual pathways. Our results provide valuable new insights into the brain representations of objects and their various physical attributes in the context of naturalistic vision.

Visual mental imagery: Evidence for a heterarchical neural architecture

Theories of Visual Mental Imagery (VMI) emphasize the processes of retrieval, modification, and recombination of sensory information from long-term memory. Yet, only few studies have focused on the behavioral mechanisms and neural correlates supporting VMI of stimuli from different semantic domains. Therefore, we currently have a limited understanding of how the brain generates and maintains mental representations of colors, faces, shapes - to name a few. Such an undetermined scenario renders unclear the organizational structure of neural circuits supporting VMI, including the role of the early visual cortex. We aimed to fill this gap by reviewing the scientific literature of five semantic domains: visuospatial, face, colors, shapes, and letters imagery. Linking theory to evidence from over 60 different experimental designs, this review highlights three main points. First, there is no consistent activity in the early visual cortex across all VMI domains, contrary to the prediction of the dominant model. Second, there is consistent activity of the frontoparietal networks and the left hemisphere's fusiform gyrus during voluntary VMI irrespective of the semantic domain investigated. We propose that these structures are part of a domain-general VMI sub-network. Third, domain-specific information engages specific regions of the ventral and dorsal cortical visual pathways. These regions partly overlap with those found in visual perception studies (e.g., fusiform face area for faces imagery; lingual gyrus for color imagery). Altogether, the reviewed evidence suggests the existence of domain-general and domain-specific mechanisms of VMI selectively engaged by stimulus-specific properties (e.g., colors or faces). These mechanisms would be supported by an organizational structure mixing vertical and horizontal connections (heterarchy) between sub-networks for specific stimulus domains. Such a heterarchical organization of VMI makes different predictions from current models of VMI as reversed perception. Our conclusions set the stage for future research, which should aim to characterize the spatiotemporal dynamics and interactions among key regions of this architecture giving rise to visual mental images.

What Does the Human Olfactory System Do, and How Does It Do It?

Historically, the human sense of smell has been regarded as the odd stepchild of the senses, especially compared to the sensory bravado of seeing, touching, and hearing. The idea that the human olfaction has little to contribute to our experience of the world is commonplace, though with the emergence of COVID-19 there has rather been a sea change in this understanding. An ever increasing body of work has convincingly highlighted the keen capabilities of the human nose and the sophistication of the human olfactory system. Here, we provide a concise overview of the neuroscience of human olfaction spanning the last 10-15 years, with focus on the peripheral and central mechanisms that underlie how odor information is processed, packaged, parceled, predicted, and perturbed to serve odor-guided behaviors. We conclude by offering some guideposts for harnessing the next decade of olfactory research in all its shapes and forms.

Ventral and Dorsal Stream EEG Channels: Key Features for EEG-Based Object Recognition and Identification

Object recognition and object identification are multifaceted cognitive operations that require various brain regions to synthesize and process information. Prior research has evidenced the activity of both visual and temporal cortices during these tasks. Notwithstanding their similarities, object recognition and identification are recognized as separate brain functions. Drawing from the two-stream hypothesis, our investigation aims to understand whether the channels within the ventral and dorsal streams contain pertinent information for effective model learning regarding object recognition and identification tasks. By utilizing the data we collected during the object recognition and identification experiment, we scrutinized EEGNet models, trained using channels that replicate the two-stream hypothesis pathways, against a model trained using all available channels. The outcomes reveal that the model trained solely using the temporal region delivered a high accuracy level in classifying four distinct object categories. Specifically, the object recognition and object identification models achieved an accuracy of 89% and 85%, respectively. By incorporating the channels that mimic the ventral stream, the model's accuracy was further improved, with the object recognition model and object identification model achieving an accuracy of 95% and 94%, respectively. Furthermore, the Grad-CAM result of the trained models revealed a significant contribution from the ventral and dorsal stream channels toward the training of the EEGNet model. The aim of our study is to pinpoint the optimal channel configuration that provides a swift and accurate brain-computer interface system for object recognition and identification.

A review of the impairments, preserved visual functions, and neuropathology in 21 patients with visual form agnosia - A unique defect with line drawings

We present a comprehensive review of the rare syndrome visual form agnosia (VFA). We begin by documenting its history, including the origins of the term, and the first case study labelled as VFA. The defining characteristics of the syndrome, as others have previously defined it, are then described. The impairments, preserved aspects of visual perception, and areas of brain damage in 21 patients who meet these defining characteristics are described in detail, including which tests were used to verify the presence or absence of key symptoms. From this, we note important similarities along with notable areas of divergence between patients. Damage to the occipital lobe (20/21), an inability to recognise line drawings (19/21), preserved colour vision (14/21), and visual field defects (16/21) were areas of consistency across most cases. We found it useful to distinguish between shape and form as distinct constructs when examining perceptual abilities in VFA patients. Our observations suggest that these patients often exhibit difficulties in processing simplified versions of form. Deficits in processing orientation and size were uncommon. Motion perception and visual imagery were not widely tested for despite being typically cited as defining features of the syndrome - although in the sample described, motion perception was never found to be a deficit. Moreover, problems with vision (e.g., poor visual acuity and the presence of hemianopias/scotomas in the visual fields) are more common than we would have thought and may also contribute to perceptual impairments in patients with VFA. We conclude that VFA is a perceptual disorder where the visual system has a reduced ability to synthesise lines together for the purposes of making sense of what images represent holistically.

Distinct Patterns of Connectivity between Brain Regions Underlie the Intra-Modal and Cross-Modal Value-Driven Modulations of the Visual Cortex

Past reward associations may be signaled from different sensory modalities; however, it remains unclear how different types of reward-associated stimuli modulate sensory perception. In this human fMRI study (female and male participants), a visual target was simultaneously presented with either an intra- (visual) or a cross-modal (auditory) cue that was previously associated with rewards. We hypothesized that, depending on the sensory modality of the cues, distinct neural mechanisms underlie the value-driven modulation of visual processing. Using a multivariate approach, we confirmed that reward-associated cues enhanced the target representation in early visual areas and identified the brain valuation regions. Then, using an effective connectivity analysis, we tested three possible patterns of connectivity that could underlie the modulation of the visual cortex: a direct pathway from the frontal valuation areas to the visual areas, a mediated pathway through the attention-related areas, and a mediated pathway that additionally involved sensory association areas. We found evidence for the third model demonstrating that the reward-related information in both sensory modalities is communicated across the valuation and attention-related brain regions. Additionally, the superior temporal areas were recruited when reward was cued cross-modally. The strongest dissociation between the intra- and cross-modal reward-driven effects was observed at the level of the feedforward and feedback connections of the visual cortex estimated from the winning model. These results suggest that, in the presence of previously rewarded stimuli from different sensory modalities, a combination of domain-general and domain-specific mechanisms are recruited across the brain to adjust the visual perception.SIGNIFICANCE STATEMENT Reward has a profound effect on perception, but it is not known whether shared or disparate mechanisms underlie the reward-driven effects across sensory modalities. In this human fMRI study, we examined the reward-driven modulation of the visual cortex by visual (intra-modal) and auditory (cross-modal) reward-associated cues. Using a model-based approach to identify the most plausible pattern of inter-regional effective connectivity, we found that higher-order areas involved in the valuation and attentional processing were recruited by both types of rewards. However, the pattern of connectivity between these areas and the early visual cortex was distinct between the intra- and cross-modal rewards. This evidence suggests that, to effectively adapt to the environment, reward signals may recruit both domain-general and domain-specific mechanisms.

Creativity at rest: Exploring functional network connectivity of creative experts

The neuroscience of creativity seeks to disentangle the complex brain processes that underpin the generation of novel ideas. Neuroimaging studies of functional connectivity, particularly functional magnetic resonance imaging (fMRI), have revealed individual differences in brain network organization associated with creative ability; however, much of the extant research is limited to laboratory-based divergent thinking measures. To overcome these limitations, we compare functional brain connectivity in a cohort of creative experts (n = 27) and controls (n = 26) and examine links with creative behavior. First, we replicate prior findings showing reduced connectivity in visual cortex related to higher creative performance. Second, we examine whether this result is driven by integrated or segregated connectivity. Third, we examine associations between functional connectivity and vivid distal simulation separately in creative experts and controls. In accordance with past work, our results show reduced connectivity to the primary visual cortex in creative experts at rest. Additionally, we observe a negative association between distal simulation vividness and connectivity to the lateral visual cortex in creative experts. Taken together, these results highlight connectivity profiles of highly creative people and suggest that creative thinking may be related to, though not fully redundant with, the ability to vividly imagine the future.

The Architecture of Object-Based Attention

The allocation of attention to objects raises several intriguing questions: What are objects, how does attention access them, what anatomical regions are involved? Here, we review recent progress in the field to determine the mechanisms underlying object-based attention. First, findings from unconscious priming and cueing suggest that the preattentive targets of object-based attention can be fully developed object representations that have reached the level of identity. Next, the control of object-based attention appears to come from ventral visual areas specialized in object analysis that project downward to early visual areas. How feedback from object areas can accurately target the object's specific locations and features is unknown but recent work in autoencoding has made this plausible. Finally, we suggest that the three classic modes of attention may not be as independent as is commonly considered, and instead could all rely on object-based attention. Specifically, studies show that attention can be allocated to the separated members of a group-without affecting the space between them-matching the defining property of feature-based attention. At the same time, object-based attention directed to a single small item has the properties of space-based attention. We outline the architecture of object-based attention, the novel predictions it brings, and discuss how it works in parallel with other attention pathways.

Backward masking implicates cortico-cortical recurrent processes in convex figure context effects and cortico-thalamic recurrent processes in resolving figure-ground ambiguity

Introduction

Previous experiments purportedly showed that image-based factors like convexity were sufficient for figure assignment. Recently, however, we found that the probability of perceiving a figure on the convex side of a central border was only slightly higher than chance for two-region displays and increased with the number of display regions; this increase was observed only when the concave regions were homogeneously colored. These convex figure context effects (CEs) revealed that figure assignment in these classic displays entails more than a response to local convexity. A Bayesian observer replicated the convex figure CEs using both a convexity object prior and a new, homogeneous background prior and made the novel prediction that the classic displays in which both the convex and concave regions were homogeneous were ambiguous during perceptual organization.

Methods

Here, we report three experiments investigating the proposed ambiguity and examining how the convex figure CEs unfold over time with an emphasis on whether they entail recurrent processing. Displays were shown for 100 ms followed by pattern masks after ISIs of 0, 50, or 100 ms. The masking conditions were designed to add noise to recurrent processing and therefore to delay the outcome of processes in which they play a role. In Exp. 1, participants viewed two- and eight-region displays with homogeneous convex regions (homo-convex displays; the putatively ambiguous displays). In Exp. 2, participants viewed putatively unambiguous hetero-convex displays. In Exp. 3, displays and masks were presented to different eyes, thereby delaying mask interference in the thalamus for up to 100 ms.

Results and discussion

The results of Exps. 1 and 2 are consistent with the interpretation that recurrent processing is involved in generating the convex figure CEs and resolving the ambiguity of homo-convex displays. The results of Exp. 3 suggested that corticofugal recurrent processing is involved in resolving the ambiguity of homo-convex displays and that cortico-cortical recurrent processes play a role in generating convex figure CEs and these two types of recurrent processes operate in parallel. Our results add to evidence that perceptual organization evolves dynamically and reveal that stimuli that seem unambiguous can be ambiguous during perceptual organization.

Perception and Memory in the Ventral Visual Stream and Medial Temporal Lobe

Perception and memory are traditionally thought of as separate cognitive functions, supported by distinct brain regions. The canonical perspective is that perceptual processing of visual information is supported by the ventral visual stream, whereas long-term declarative memory is supported by the medial temporal lobe. However, this modular framework cannot account for the increasingly large body of evidence that reveals a role for early visual areas in long-term recognition memory and a role for medial temporal lobe structures in high-level perceptual processing. In this article, we review relevant research conducted in humans, nonhuman primates, and rodents. We conclude that the evidence is largely inconsistent with theoretical proposals that draw sharp functional boundaries between perceptual and memory systems in the brain. Instead, the weight of the empirical findings is best captured by a representational-hierarchical model that emphasizes differences in content, rather than in cognitive processes within the ventral visual stream and medial temporal lobe.

The effect of context congruency on fMRI repetition suppression for objects

The recognition of objects is strongly facilitated when they are presented in the context of other objects (Biederman, 1972). Such contexts facilitate perception and induce expectations of context-congruent objects (Trapp and Bar, 2015). The neural mechanisms underlying these facilitatory effects of context on object processing, however, are not yet fully understood. In the present study, we investigate how context-induced expectations affect subsequent object processing. We used functional magnetic resonance imaging and measured repetition suppression as a proxy for prediction error processing. Participants viewed pairs of alternating or repeated object images which were preceded by context-congruent, context-incongruent or neutral cues. We found a stronger repetition suppression in congruent as compared to incongruent or neutral cues in the object sensitive lateral occipital cortex. Interestingly, this stronger effect was driven by enhanced responses to alternating stimulus pairs in the congruent contexts, rather than by suppressed responses to repeated stimulus pairs, which emphasizes the contribution of surprise-related response enhancement for the context modulation on RS when expectations are violated. In addition, in the congruent condition, we discovered significant functional connectivity between object-responsive and frontal cortical regions, as well as between object-responsive regions and the fusiform gyrus. Our findings indicate that prediction errors, reflected in enhanced brain responses to violated contextual expectations, underlie the facilitating effect of context during object perception.

Standardised images of novel objects created with generative adversarial networks

An enduring question in cognitive science is how perceptually novel objects are processed. Addressing this issue has been limited by the absence of a standardised set of object-like stimuli that appear realistic, but cannot possibly have been previously encountered. To this end, we created a dataset, at the core of which are images of 400 perceptually novel objects. These stimuli were created using Generative Adversarial Networks that integrated features of everyday stimuli to produce a set of synthetic objects that appear entirely plausible, yet do not in fact exist. We curated an accompanying dataset of 400 familiar stimuli, which were matched in terms of size, contrast, luminance, and colourfulness. For each object, we quantified their key visual properties (edge density, entropy, symmetry, complexity, and spectral signatures). We also confirmed that adult observers (N = 390) perceive the novel objects to be less familiar, yet similarly engaging, relative to the familiar objects. This dataset serves as an open resource to facilitate future studies on visual perception.

Comparing the Dominance of Color and Form Information across the Human Ventral Visual Pathway and Convolutional Neural Networks

Color and form information can be decoded in every region of the human ventral visual hierarchy, and at every layer of many convolutional neural networks (CNNs) trained to recognize objects, but how does the coding strength of these features vary over processing? Here, we characterize for these features both their absolute coding strength-how strongly each feature is represented independent of the other feature-and their relative coding strength-how strongly each feature is encoded relative to the other, which could constrain how well a feature can be read out by downstream regions across variation in the other feature. To quantify relative coding strength, we define a measure called the form dominance index that compares the relative influence of color and form on the representational geometry at each processing stage. We analyze brain and CNN responses to stimuli varying based on color and either a simple form feature, orientation, or a more complex form feature, curvature. We find that while the brain and CNNs largely differ in how the absolute coding strength of color and form vary over processing, comparing them in terms of their relative emphasis of these features reveals a striking similarity: For both the brain and for CNNs trained for object recognition (but not for untrained CNNs), orientation information is increasingly de-emphasized, and curvature information is increasingly emphasized, relative to color information over processing, with corresponding processing stages showing largely similar values of the form dominance index.

The neural correlates of domain-general visual ability

People vary in their general ability to compare, identify, and remember objects. Research using latent variable modeling identifies a domain-general visual recognition ability (called o) that reflects correlations among different visual tasks and categories. We measure associations between a psychometrically-sensitive measure of o and a neurometrically-sensitive measure of visual sensitivity to shape. We report evidence for distributed neural correlates of o using functional and anatomical regions-of-interest (ROIs) as well as whole brain analyses. Neural selectivity to shape is associated with o in several regions of the ventral pathway, as well as additional foci in parietal and premotor cortex. Multivariate analyses suggest the distributed effects in ventral cortex reflect a common mechanism. The network of brain areas where neural selectivity predicts o is similar to that evoked by the most informative features for object recognition in prior work, showing convergence of 2 different approaches on identifying areas that support the best object recognition performance. Because o predicts performance across many visual tasks for both novel and familiar objects, we propose that o could predict the magnitude of neural changes in task-relevant areas following experience with specific task and object category.

Seeing and extrapolating motion trajectories share common informative activation patterns in primary visual cortex

The natural environment is dynamic and moving objects become constantly occluded, engaging the brain in a challenging completion process to estimate where and when the object might reappear. Although motion extrapolation is critical in daily life-imagine crossing the street while an approaching car is occluded by a larger standing vehicle-its neural underpinnings are still not well understood. While the engagement of low-level visual cortex during dynamic occlusion has been postulated, most of the previous group-level fMRI-studies failed to find evidence for an involvement of low-level visual areas during occlusion. In this fMRI-study, we therefore used individually defined retinotopic maps and multivariate pattern analysis to characterize the neural basis of visible and occluded changes in motion direction in humans. To this end, participants learned velocity-direction change pairings (slow motion-upwards; fast motion-downwards or vice versa) during a training phase without occlusion and judged the change in stimulus direction, based on its velocity, during a following test phase with occlusion. We find that occluded motion direction can be predicted from the activity patterns during visible motion within low-level visual areas, supporting the notion of a mental representation of motion trajectory in these regions during occlusion.

Context-based modulations of 3D vision are expertise dependent

An object's identity can influence depth-position judgments. The mechanistic underpinnings underlying this phenomenon are largely unknown. Here, we asked whether context-dependent modulations of stereoscopic depth perception are expertise dependent. In 2 experiments, we tested whether training that attaches meaning (i.e. classification labels) to otherwise novel, stereoscopically presented objects changes observers' sensitivity for judging their depth position. In Experiment 1, observers were randomly assigned to 3 groups: a Greeble-classification training group, an orientation-discrimination training group, or a no-training group, and were tested on their stereoscopic depth sensitivity before and after training. In Experiment 2, participants were tested before and after training while fMRI responses were concurrently imaged. Behaviorally, stereoscopic performance was significantly better following Greeble-classification (but not orientation-discrimination, or no-) training. Using the fMRI data, we trained support vector machines to predict whether the data were from the pre- or post-training sessions. Results indicated that classification accuracies in V4 were higher for the Greeble-classification group as compared with the orientation-discrimination group for which accuracies were at chance level. Furthermore, classification accuracies in V4 were negatively correlated with response times for Greeble identification. We speculate that V4 is implicated in an expertise-dependent, object-tuning manner that allows it to better guide stereoscopic depth retrieval.

From pictures to reality: modelling the phenomenology and psychophysics of 3D perception

The dominant inferential approach to human 3D perception assumes a model of spatial encoding based on a physical description of objects and space. Prevailing models based on this physicalist approach assume that the visual system infers an objective, unitary and mostly veridical representation of the external world. However, careful consideration of the phenomenology of 3D perception challenges these assumptions. I review important aspects of phenomenology, psychophysics and neurophysiology which suggest that human visual perception of 3D  objects and space is underwritten by distinct and dissociated spatial encodings that are optimized for specific regions of space. Specifically, I argue that 3D perception is underwritten by at least three distinct encodings for (1) egocentric distance perception at the ambulatory scale, (2) exocentric distance (scaled depth) perception optimized for near space, and (3) perception of object shape and layout (unscaled depth). This tripartite division can more satisfactorily account for the phenomenology, psychophysics and adaptive logic of human 3D perception. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Understanding Human Object Vision: A Picture Is Worth a Thousand Representations

Objects are the core meaningful elements in our visual environment. Classic theories of object vision focus upon object recognition and are elegant and simple. Some of their proposals still stand, yet the simplicity is gone. Recent evolutions in behavioral paradigms, neuroscientific methods, and computational modeling have allowed vision scientists to uncover the complexity of the multidimensional representational space that underlies object vision. We review these findings and propose that the key to understanding this complexity is to relate object vision to the full repertoire of behavioral goals that underlie human behavior, running far beyond object recognition. There might be no such thing as core object recognition, and if it exists, then its importance is more limited than traditionally thought.

Cortical Deficits are Correlated with Impaired Stereopsis in Patients with Strabismus

In this study, we explored the neural mechanism underlying impaired stereopsis and possible functional plasticity after strabismus surgery. We enrolled 18 stereo-deficient patients with intermittent exotropia before and after surgery, along with 18 healthy controls. Functional magnetic resonance imaging data were collected when participants viewed three-dimensional stimuli. Compared with controls, preoperative patients showed hypoactivation in higher-level dorsal (visual and parietal) areas and ventral visual areas. Pre- and postoperative activation did not significantly differ in patients overall; patients with improved stereopsis showed stronger postoperative activation than preoperative activation in the right V3A and left intraparietal sulcus. Worse stereopsis and fusional control were correlated with preoperative hypoactivation, suggesting that cortical deficits along the two streams might reflect impaired stereopsis in intermittent exotropia. The correlation between improved stereopsis and activation in the right V3A after surgery indicates that functional plasticity may underlie the improvement of stereopsis. Thus, additional postoperative strategies are needed to promote functional plasticity and enhance the recovery of stereopsis.

Emotional learning retroactively promotes memory integration through rapid neural reactivation and reorganization

Neutral events preceding emotional experiences can be better remembered, likely by assigning them as significant to guide possible use in future. Yet, the neurobiological mechanisms of how emotional learning enhances memory for past mundane events remain unclear. By two behavioral studies and one functional magnetic resonance imaging study with an adapted sensory preconditioning paradigm, we show rapid neural reactivation and connectivity changes underlying emotion-charged retroactive memory enhancement. Behaviorally, emotional learning retroactively enhanced initial memory for neutral associations across the three studies. Neurally, emotional learning potentiated trial-specific reactivation of overlapping neural traces in the hippocampus and stimulus-relevant neocortex. It further induced rapid hippocampal-neocortical functional reorganization supporting such retroactive memory benefit, as characterized by enhanced hippocampal-neocortical coupling modulated by the amygdala during emotional learning, and a shift of hippocampal connectivity from stimulus-relevant neocortex to distributed transmodal prefrontal-parietal areas at post-learning rests. Together, emotional learning retroactively promotes memory integration for past neutral events through stimulating trial-specific reactivation of overlapping representations and reorganization of associated memories into an integrated network to foster its priority for future use.

Independent repetition suppression in macaque area V2 and inferotemporal cortex

When a complexly structured natural image is presented twice in succession, first as adapter and then as test, neurons in area TE of macaque inferotemporal cortex exhibit repetition suppression, responding less strongly to the second presentation than to the first. This phenomenon, which has been studied primarily in TE, might plausibly be argued to arise in TE because TE neurons respond selectively to complex images and thus carry information adequate for determining whether an image is or is not a repeat. However, the idea has never been put to a direct test. To resolve this issue, we monitored neuronal responses to sequences of complex natural images under identical conditions in areas V2 and TE. We found that repetition suppression occurs in both areas. Moreover, in each area, suppression takes the form of a dynamic alteration whereby the initial peak of excitation is followed by a trough and then a rebound of firing rate. To assess whether repetition suppression in either area is transmitted from the other area, we analyzed the timing of the phenomenon and its degree of spatial generalization. Suppression occurs at shorter latency in V2 than in TE. Therefore it is not simply fed back from TE. Suppression occurs in TE but not in V2 under conditions in which the test and adapter are presented in different visual field quadrants. Therefore it is not simply fed forward from V2. We conclude that repetition suppression occurs independently in V2 and TE.NEW & NOTEWORTHY When a complexly structured natural image is presented twice in rapid succession, neurons in inferotemporal area TE exhibit repetition suppression, responding less strongly to the second than to the first presentation. We have explored whether this phenomenon is confined to high-order areas where neurons respond selectively to such images and thus carry information relevant to recognizing a repeat. We have found surprisingly that repetition suppression occurs even in low-order visual area V2.

Does the brain's ventral visual pathway compute object shape?

A rich behavioral literature has shown that human object recognition is supported by a representation of shape that is tolerant to variations in an object's appearance. Such 'global' shape representations are achieved by describing objects via the spatial arrangement of their local features, or structure, rather than by the appearance of the features themselves. However, accumulating evidence suggests that the ventral visual pathway - the primary substrate underlying object recognition - may not represent global shape. Instead, ventral representations may be better described as a basis set of local image features. We suggest that this evidence forces a reevaluation of the role of the ventral pathway in object perception and posits a broader network for shape perception that encompasses contributions from the dorsal pathway.

Scene-selective brain regions respond to embedded objects of a scene

Objects are fundamental to scene understanding. Scenes are defined by embedded objects and how we interact with them. Paradoxically, scene processing in the brain is typically discussed in contrast to object processing. Using the BOLD5000 dataset (Chang et al., 2019), we examined whether objects within a scene predicted the neural representation of scenes, as measured by functional magnetic resonance imaging in humans. Stimuli included 1,179 unique scenes across 18 semantic categories. Object composition of scenes were compared across scene exemplars in different semantic scene categories, and separately, in exemplars of the same scene category. Neural representations in scene- and object-preferring brain regions were significantly related to which objects were in a scene, with the effect at times stronger in the scene-preferring regions. The object model accounted for more variance when comparing scenes within the same semantic category to scenes from different categories. Here, we demonstrate the function of scene-preferring regions includes the processing of objects. This suggests visual processing regions may be better characterized by the processes, which are engaged when interacting with the stimulus kind, such as processing groups of objects in scenes, or processing a single object in our foreground, rather than the stimulus kind itself.

Assessing the ecological validity of numerosity-selective neuronal populations with real-world natural scenes

Animals and humans are able to quickly and effortlessly estimate the number of items in a set: their numerosity. Numerosity perception is thought to be critical to behavior, from feeding to escaping predators to human mathematical cognition. Virtually, all scientific studies on numerosity mechanisms use well controlled but artificial stimuli to isolate the numerosity dimension from other physical quantities. Here, we probed the ecological validity of these artificial stimuli and evaluate whether an important component in numerosity processing, the numerosity-selective neural populations, also respond to numerosity of items in real-world natural scenes. Using 7T MRI and natural images from a wide range of categories, we provide evidence that the numerosity-tuned neuronal populations show numerosity-selective responses when viewing images from a real-world natural scene. Our findings strengthen the role of numerosity-selective neurons in numerosity perception and provide an important link to their function in numerosity perception in real-world settings.

Association of Nucleus Basalis of Meynert Functional Connectivity and Cognition in Idiopathic Rapid-Eye-Movement Sleep Behavior Disorder

BACKGROUND AND PURPOSE

Cognitive impairments are common in isolated rapid-eye-movement sleep behavior disorder (iRBD), in which the cholinergic system may play an important role. This study aimed to characterize the cortical cholinergic activity using resting-state functional connectivity (FC) of the nucleus basalis of Meynert (NBM) according to the cognitive status of iRBD patients.

METHODS

In this cross-sectional study, 33 patients with polysomnography-confirmed iRBD and 20 controls underwent neuropsychological evaluations and resting-state functional magnetic resonance imaging. Thirteen of the iRBD patients had mild cognitive impairment (iRBD-MCI), and the others were age-matched patients with normal cognition (iRBD-NC). The seed-to-voxel NBM-cortical FC was compared among the patients with iRBD-MCI, patients with iRBD-NC, and controls. Correlations between average values of significant clusters and cognitive function scores were calculated in the patients with iRBD.

RESULTS

There were group differences in the FC of the NBM with the left lateral occipital cortex and lingual gyrus (adjusted for age, sex, and education level). The strength of FC was lower in the iRBD-MCI group than in the iRBD-NC and control groups (each post-hoc p<0.001). The average NBM-lateral occipital cortex FC was positively correlated with the memory-domain score in iRBD patients.

CONCLUSIONS

The results obtained in this study support that cortical cholinergic activity is impaired in iRBD patients with MCI. FC between NBM and posterior regions may play a central role in the cognitive function of these patients.