Topographic connectivity reveals task-dependent retinotopic processing throughout the human brain

Testing the top-down feedback in the central visual field using the reversed depth illusion

In a new framework to understand vision, an information bottleneck impoverishes visual input information downstream of the primary visual cortex along the visual pathway; to aid ongoing visual recognition given the bottleneck, feedback from downstream to upstream visual stages queries for additional information. According to the central-peripheral dichotomy theory, this feedback is primarily directed to the central, rather than the peripheral, visual field. Counterintuitively, this theory predicts illusions visible only in the peripheral visual field, which lacks the feedback query to veto the illusions arising from misleading and impoverished feedforward signals. A paradigmatic example is the predicted and confirmed reversed depth illusion in random-dot stereograms. This theory further predicts that disrupting the feedback renders this illusion visible in the central visual field. We test and confirm this prediction using visual backward masking to disrupt the feedback. This feedback privilege for the central visual field underpins visual understanding through analysis-by-synthesis.

Movies reveal the fine-grained organization of infant visual cortex

Studying infant minds with movies is a promising way to increase engagement relative to traditional tasks. However, the spatial specificity and functional significance of movie-evoked activity in infants remains unclear. Here, we investigated what movies can reveal about the organization of the infant visual system. We collected fMRI data from 15 awake infants and toddlers aged 5-23 months who attentively watched a movie. The activity evoked by the movie reflected the functional profile of visual areas. Namely, homotopic areas from the two hemispheres responded similarly to the movie, whereas distinct areas responded dissimilarly, especially across dorsal and ventral visual cortex. Moreover, visual maps that typically require time-intensive and complicated retinotopic mapping could be predicted, albeit imprecisely, from movie-evoked activity in both data-driven analyses (i.e. independent component analysis) at the individual level and by using functional alignment into a common low-dimensional embedding to generalize across participants. These results suggest that the infant visual system is already structured to process dynamic, naturalistic information and that fine-grained cortical organization can be discovered from movie data.

A double dissociation between semantic and spatial cognition in visual to default network pathways

Processing pathways between sensory and default mode network (DMN) regions support recognition, navigation, and memory but their organisation is not well understood. We show that functional subdivisions of visual cortex and DMN sit at opposing ends of parallel streams of information processing that support visually mediated semantic and spatial cognition, providing convergent evidence from univariate and multivariate task responses, intrinsic functional and structural connectivity. Participants learned virtual environments consisting of buildings populated with objects, drawn from either a single semantic category or multiple categories. Later, they made semantic and spatial context decisions about these objects and buildings during functional magnetic resonance imaging. A lateral ventral occipital to fronto-temporal DMN pathway was primarily engaged by semantic judgements, while a medial visual to medial temporal DMN pathway supported spatial context judgements. These pathways had distinctive locations in functional connectivity space: the semantic pathway was both further from unimodal systems and more balanced between visual and auditory-motor regions compared with the spatial pathway. When semantic and spatial context information could be integrated (in buildings containing objects from a single category), regions at the intersection of these pathways responded, suggesting that parallel processing streams interact at multiple levels of the cortical hierarchy to produce coherent memory-guided cognition.

The Scope and Limits of Fine-Grained Image and Category Information in the Ventral Visual Pathway

Humans can easily abstract incoming visual information into discrete semantic categories. Previous research employing functional MRI (fMRI) in humans has identified cortical organizing principles that allow not only for coarse-scale distinctions such as animate versus inanimate objects but also more fine-grained distinctions at the level of individual objects. This suggests that fMRI carries rather fine-grained information about individual objects. However, most previous work investigating fine-grained category representations either additionally included coarse-scale category comparisons of objects, which confounds fine-grained and coarse-scale distinctions, or only used a single exemplar of each object, which confounds visual and semantic information. To address these challenges, here we used multisession human fMRI (female and male) paired with a broad yet homogenous stimulus class of 48 terrestrial mammals, with two exemplars per mammal. Multivariate decoding and representational similarity analysis revealed high image-specific reliability in low- and high-level visual regions, indicating stable representational patterns at the image level. In contrast, analyses across exemplars of the same animal yielded only small effects in the lateral occipital complex (LOC), indicating rather subtle category effects in this region. Variance partitioning with a deep neural network and shape model showed that across-exemplar effects in the early visual cortex were largely explained by low-level visual appearance, while representations in LOC appeared to also contain higher category-specific information. These results suggest that representations typically measured with fMRI are dominated by image-specific visual or coarse-grained category information but indicate that commonly employed fMRI protocols may reveal subtle yet reliable distinctions between individual objects.

Retinotopic coding organizes the interaction between internally and externally oriented brain networks

The human brain seamlessly integrates internally generated thoughts with incoming sensory information, yet the networks supporting internal (default network, DN) and external (dorsal attention network, dATN) processing are traditionally viewed as antagonistic. This raises a crucial question: how does the brain integrate information between these seemingly opposed systems? Here, using precision neuroimaging methods, we show that these internal/external networks are not as dissociated as traditionally thought. Using densely-sampled 7T fMRI data, we defined individualized whole-brain networks from participants at rest and calculated the retinotopic preferences of individual voxels within these networks during an visual mapping task. We show that while the overall network activity between the DN and dATN is independent at rest, considering a latent retinotopic code reveals a complex, voxel-scale interaction stratified by visual responsiveness. Specifically, the interaction between the DN and dATN at rest is structured at the voxel-level by each voxel's retinotopic preferences, such that the spontaneous activity of voxels preferring similar visual field locations is more anti-correlated than that of voxels preferring different visual field locations. Further, this retinotopic scaffold integrates with the domain-specific preferences of subregions within these networks, enabling efficient, parallel processing of retinotopic and domain-specific information. Thus, DN and dATN are not independent at rest: voxel-scale interaction between these networks preserves and encodes information in both positive and negative BOLD responses, even in the absence of visual input or task demands. These findings suggest that retinotopic coding may serve as a fundamental organizing principle for brain-wide communication, providing a new framework for understanding how the brain balances and integrates internal cognition with external perception.

The brain hierarchically represents the past and future during multistep anticipation

Memory for temporal structure enables both planning of future events and retrospection of past events. We investigated how the brain flexibly represents extended temporal sequences into the past and future during anticipation. Participants learned sequences of environments in immersive virtual reality. Pairs of sequences had the same environments in a different order, enabling context-specific learning. During fMRI, participants anticipated upcoming environments multiple steps into the future in a given sequence. Temporal structure was represented in the hippocampus and across higher-order visual regions (1) bidirectionally, with graded representations into the past and future and (2) hierarchically, with further events into the past and future represented in successively more anterior brain regions. In hippocampus, these bidirectional representations were context-specific, and suppression of far-away environments predicted response time costs in anticipation. Together, this work sheds light on how we flexibly represent sequential structure to enable planning over multiple timescales.

Positive and Negative Retinotopic Codes in the Human Hippocampus

The hippocampus is thought to coordinate sensory-mnemonic information streams in the brain, representing both the apex of the visual processing hierarchy and the central hub of mnemonic processing. Yet, the mechanisms underlying sensory-mnemonic interactions in the hippocampus are poorly understood. Recent work in cortex suggests that a retinotopic code - typically thought to be exclusive to visual areas - may help organize internal and external information at the cortical apex via opponent interactions. Here, we leverage high-resolution 7T functional MRI to test whether a bivalent retinotopic code structures activity within the human hippocampus and mediates hippocampal-cortical interactions. In seven densely-sampled individuals, we defined the retinotopic preferences of individual voxels within the hippocampus and cortex during a visual mapping task, as well as their functional connectivity during independent runs of resting-state fixation. Our findings reveal a robust retinotopic code in the hippocampus, characterized by stable population receptive fields (pRFs) with consistent preferred visual field locations across experimental runs. Notably, this retinotopic code is comprised of roughly equal proportions of positive and negative pRFs, aligning with the hypothesized role of negative pRFs in mnemonic processing. Finally, the signed amplitude of hippocampal pRFs predicts functional connectivity between retinotopic hippocampal and cortical voxels. Taken together, these results suggest that retinotopic coding may scaffold internal mnemonic and external sensory information processing within the hippocampus, and across hippocampal-cortical interactions.

Network mechanisms of ongoing brain activity's influence on conscious visual perception

Sensory inputs enter a constantly active brain, whose state is always changing from one moment to the next. Currently, little is known about how ongoing, spontaneous brain activity participates in online task processing. We employed 7 Tesla fMRI and a threshold-level visual perception task to probe the effects of prestimulus ongoing brain activity on perceptual decision-making and conscious recognition. Prestimulus activity originating from distributed brain regions, including visual cortices and regions of the default-mode and cingulo-opercular networks, exerted a diverse set of effects on the sensitivity and criterion of conscious recognition, and categorization performance. We further elucidate the mechanisms underlying these behavioral effects, revealing how prestimulus activity modulates multiple aspects of stimulus processing in highly specific and network-dependent manners. These findings reveal heretofore unknown network mechanisms underlying ongoing brain activity's influence on conscious perception, and may hold implications for understanding the precise roles of spontaneous activity in other brain functions.

Attention spotlight in V1-based cortico-cortical interactions in human visual hierarchy

Attention is often viewed as a mental spotlight, which can be scaled like a zoom lens at specific spatial locations and features a center-surround gradient. Here, we demonstrate a neural signature of attention spotlight in signal transmission along the visual hierarchy. fMRI background connectivity analysis was performed between retinotopic V1 and downstream areas to characterize the spatial distribution of inter-areal interaction under two attentional states. We found that, compared to diffused attention, focal attention sharpened the spatial gradient in the strength of the background connectivity. Dynamic causal modeling analysis further revealed the effect of attention in both the feedback and feedforward connectivity between V1 and extrastriate cortex. In a context which induced a strong effect of crowding, the effect of attention in the background connectivity profile diminished. Our findings reveal a context-dependent attention prioritization in information transmission via modulating the recurrent processing across the early stages in human visual cortex.

Mapping the functional and structural connectivity of the scene network

The recognition and perception of places has been linked to a network of scene-selective regions in the human brain. While previous studies have focussed on functional connectivity between scene-selective regions themselves, less is known about their connectivity with other cortical and subcortical regions in the brain. Here, we determine the functional and structural connectivity profile of the scene network. We used fMRI to examine functional connectivity between scene regions and across the whole brain during rest and movie-watching. Connectivity within the scene network revealed a bias between posterior and anterior scene regions implicated in perceptual and mnemonic aspects of scene perception respectively. Differences between posterior and anterior scene regions were also evident in the connectivity with cortical and subcortical regions across the brain. For example, the Occipital Place Area (OPA) and posterior Parahippocampal Place Area (PPA) showed greater connectivity with visual and dorsal attention networks, while anterior PPA and Retrosplenial Complex showed preferential connectivity with default mode and frontoparietal control networks and the hippocampus. We further measured the structural connectivity of the scene network using diffusion tractography. This indicated both similarities and differences with the functional connectivity, highlighting biases between posterior and anterior regions, but also between ventral and dorsal scene regions. Finally, we quantified the structural connectivity between the scene network and major white matter tracts throughout the brain. These findings provide a map of the functional and structural connectivity of scene-selective regions to each other and the rest of the brain.

A retinotopic code structures the interaction between perception and memory systems

Conventional views of brain organization suggest that regions at the top of the cortical hierarchy processes internally oriented information using an abstract amodal neural code. Despite this, recent reports have described the presence of retinotopic coding at the cortical apex, including the default mode network. What is the functional role of retinotopic coding atop the cortical hierarchy? Here we report that retinotopic coding structures interactions between internally oriented (mnemonic) and externally oriented (perceptual) brain areas. Using functional magnetic resonance imaging, we observed robust inverted (negative) retinotopic coding in category-selective memory areas at the cortical apex, which is functionally linked to the classic (positive) retinotopic coding in category-selective perceptual areas in high-level visual cortex. These functionally linked retinotopic populations in mnemonic and perceptual areas exhibit spatially specific opponent responses during both bottom-up perception and top-down recall, suggesting that these areas are interlocked in a mutually inhibitory dynamic. These results show that retinotopic coding structures interactions between perceptual and mnemonic neural systems, providing a scaffold for their dynamic interaction.

Organization of pRF size along the AP axis of the hippocampus and adjacent medial temporal cortex is related to specialization for scenes versus faces

The hippocampus is largely recognized for its integral contributions to memory processing. By contrast, its role in perceptual processing remains less clear. Hippocampal properties vary along the anterior-posterior (AP) axis. Based on past research suggesting a gradient in the scale of features processed along the AP extent of the hippocampus, the representations have been proposed to vary as a function of granularity along this axis. One way to quantify such granularity is with population receptive field (pRF) size measured during visual processing, which has so far received little attention. In this study, we compare the pRF sizes within the hippocampus to its activation for images of scenes versus faces. We also measure these functional properties in surrounding medial temporal lobe (MTL) structures. Consistent with past research, we find pRFs to be larger in the anterior than in the posterior hippocampus. Critically, our analysis of surrounding MTL regions, the perirhinal cortex, entorhinal cortex, and parahippocampal cortex shows a similar correlation between scene sensitivity and larger pRF size. These findings provide conclusive evidence for a tight relationship between the pRF size and the sensitivity to image content in the hippocampus and adjacent medial temporal cortex.

Systematic transition from boundary extension to contraction along an object-to-scene continuum

After viewing a picture of an environment, our memory of it typically extends beyond what was presented, a phenomenon referred to as boundary extension. But, sometimes memory errors show the opposite pattern-boundary contraction-and the relationship between these phenomena is controversial. We constructed virtual three-dimensional environments and created a series of views at different distances, from object close-ups to wide-angle indoor views, and tested for memory errors along this object-to-scene continuum. Boundary extension was evident for close-scale views and transitioned parametrically to boundary contraction for far-scale views. However, this transition point was not tied to a specific position in the environment (e.g., the point of reachability). Instead, it tracked with judgments of the best-looking view of the environment, in both rich-object and low-object environments. We offer a dynamic-tension account, where competition between object-based and scene-based affordances determines whether a view will extend or contract in memory. This study demonstrates that boundary extension and boundary contraction are not two separate phenomena but rather two parts of a continuum, suggesting a common underlying mechanism. The transition point between the two is not fixed but depends on the observer's judgment of the best-looking view of the environment. These findings provide new insights into how we perceive and remember a view of environment.

A retinotopic code structures the interaction between perception and memory systems

Conventional views of brain organization suggest that the cortical apex processes internally-oriented information using an abstract, amodal neural code. Yet, recent reports have described the presence of retinotopic coding at the cortical apex, including the default mode network. What is the functional role of retinotopic coding atop the cortical hierarchy? Here, we report that retinotopic coding structures interactions between internally-oriented (mnemonic) and externally-oriented (perceptual) brain areas. Using fMRI, we observed robust, inverted (negative) retinotopic coding in category-selective memory areas at the cortical apex, which is functionally linked to the classic (positive) retinotopic coding in category-selective perceptual areas in high-level visual cortex. Specifically, these functionally-linked retinotopic populations in mnemonic and perceptual areas exhibit spatially-specific opponent responses during both bottom-up perception and top-down recall, suggesting that these areas are interlocked in a mutually-inhibitory dynamic. Together, these results show that retinotopic coding structures interactions between perceptual and mnemonic neural systems, thereby scaffolding their dynamic interaction.

Topographical Association Between Left Ventricular Strain and Brain Lesions in Patients With Acute Ischemic Stroke and Normal Cardiac Function

Background Although it is well known that the disordered brain provokes cardiac autonomic dysfunction, the detailed location of brain lesions related to cardiac function warrants further investigation. We aimed to elucidate the brain lesions topographically associated with left ventricular (LV) systolic function measured by myocardial strain in patients with acute ischemic stroke without preexisting primary cardiac dysfunction by using support vector regression lesion-symptom mapping. Methods and Results Subjects were those with LV ejection fraction of 50% or more among patients with acute ischemic stroke registered in the Samsung Medical Center stroke registry between 2016 and 2017. To evaluate LV systolic performance and contractility, we measured LV ejection fraction and LV global and regional longitudinal strain using 2-dimensional speckle-tracking echocardiography. The association between stroke lesion location and cardiac strain was assessed using support vector regression lesion-symptom mapping. Of a total of 776 patients, 286 subjects (mean age of 67.0 years, 65.4% men) were finally enrolled in this study. The mean global longitudinal strain was -17.0±3.4%, and the mean LV ejection fraction was 64.7±5.7%. The support vector regression lesion-symptom mapping analysis revealed that the right insula and peri-insular regions and left parietal cortex were associated with impaired LV global longitudinal strain in patients with acute ischemic stroke. In addition, impaired regional longitudinal strain showed topographical associations with these regions. Conclusions This study suggests that brain lesions in the right insula and peri-insular regions and left parietal cortex are topographically associated with impaired LV strain in patients with acute ischemic stroke without preexisting cardiac dysfunction.

Shape, perspective, and what is and is not perceived: Comment on Morales, Bax, and Firestone (2020)

Psychology and philosophy have long reflected on the role of perspective in vision. Since the dawn of modern vision science-roughly, since Helmholtz in the late 1800s-scientific explanations in vision have focused on understanding the computations that transform the sensed retinal image into percepts of the three-dimensional environment. The standard view in the science is that distal properties-viewpoint-independent properties of the environment (object shape) and viewpoint-dependent relational properties (3D orientation relative to the viewer)-are perceptually represented and that properties of the proximal stimulus (in vision, the retinal image) are not. This view is woven into the nature of scientific explanation in perceptual psychology, and has guided impressive advances over the past 150 years. A recently published article suggests that in shape perception, the standard view must be revised. It argues, on the basis of new empirical data, that a new entity-perspectival shape-should be introduced into scientific explanations of shape perception. Specifically, the article's centrally advertised claim is that, in addition to distal shape, perspectival shape is perceived. We argue that this claim rests on a series of mistakes. Problems in experimental design entail that the article provides no empirical support for any claims regarding either perspective or the perception of shape. There are further problems in scientific reasoning and conceptual development. Detailing these criticisms and explaining how science treats these issues are meant to clarify method and theory, and to improve exchanges between the science and philosophy of perception. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

The entorhinal-DG/CA3 pathway in the medial temporal lobe retains visual working memory of a simple surface feature

Classic models consider working memory (WM) and long-term memory as distinct mental faculties that are supported by different neural mechanisms. Yet, there are significant parallels in the computation that both types of memory require. For instance, the representation of precise item-specific memory requires the separation of overlapping neural representations of similar information. This computation has been referred to as pattern separation, which can be mediated by the entorhinal-DG/CA3 pathway of the medial temporal lobe (MTL) in service of long-term episodic memory. However, although recent evidence has suggested that the MTL is involved in WM, the extent to which the entorhinal-DG/CA3 pathway supports precise item-specific WM has remained elusive. Here, we combine an established orientation WM task with high-resolution fMRI to test the hypothesis that the entorhinal-DG/CA3 pathway retains visual WM of a simple surface feature. Participants were retrospectively cued to retain one of the two studied orientation gratings during a brief delay period and then tried to reproduce the cued orientation as precisely as possible. By modeling the delay-period activity to reconstruct the retained WM content, we found that the anterior-lateral entorhinal cortex (aLEC) and the hippocampal DG/CA3 subfield both contain item-specific WM information that is associated with subsequent recall fidelity. Together, these results highlight the contribution of MTL circuitry to item-specific WM representation.

Microstructural and neurochemical plasticity mechanisms interact to enhance human perceptual decision-making

Experience and training are known to boost our skills and mold the brain's organization and function. Yet, structural plasticity and functional neurotransmission are typically studied at different scales (large-scale networks, local circuits), limiting our understanding of the adaptive interactions that support learning of complex cognitive skills in the adult brain. Here, we employ multimodal brain imaging to investigate the link between microstructural (myelination) and neurochemical (GABAergic) plasticity for decision-making. We test (in males, due to potential confounding menstrual cycle effects on GABA measurements in females) for changes in MRI-measured myelin, GABA, and functional connectivity before versus after training on a perceptual decision task that involves identifying targets in clutter. We demonstrate that training alters subcortical (pulvinar, hippocampus) myelination and its functional connectivity to visual cortex and relates to decreased visual cortex GABAergic inhibition. Modeling interactions between MRI measures of myelin, GABA, and functional connectivity indicates that pulvinar myelin plasticity interacts-through thalamocortical connectivity-with GABAergic inhibition in visual cortex to support learning. Our findings propose a dynamic interplay of adaptive microstructural and neurochemical plasticity in subcortico-cortical circuits that supports learning for optimized decision-making in the adult human brain.

Successor-like representation guides the prediction of future events in human visual cortex and hippocampus

Human agents build models of their environment, which enable them to anticipate and plan upcoming events. However, little is known about the properties of such predictive models. Recently, it has been proposed that hippocampal representations take the form of a predictive map-like structure, the so-called successor representation (SR). Here, we used human functional magnetic resonance imaging to probe whether activity in the early visual cortex (V1) and hippocampus adhere to the postulated properties of the SR after visual sequence learning. Participants were exposed to an arbitrary spatiotemporal sequence consisting of four items (A-B-C-D). We found that after repeated exposure to the sequence, merely presenting single sequence items (e.g., - B - -) resulted in V1 activation at the successor locations of the full sequence (e.g., C-D), but not at the predecessor locations (e.g., A). This highlights that visual representations are skewed toward future states, in line with the SR. Similar results were also found in the hippocampus. Moreover, the hippocampus developed a coactivation profile that showed sensitivity to the temporal distance in sequence space, with fading representations for sequence events in the more distant past and future. V1, in contrast, showed a coactivation profile that was only sensitive to spatial distance in stimulus space. Taken together, these results provide empirical evidence for the proposition that both visual and hippocampal cortex represent a predictive map of the visual world akin to the SR.

Virtual reality-based sensorimotor adaptation shapes subsequent spontaneous and naturalistic stimulus-driven brain activity

Our everyday life summons numerous novel sensorimotor experiences, to which our brain needs to adapt in order to function properly. However, tracking plasticity of naturalistic behavior and associated brain modulations is challenging. Here, we tackled this question implementing a prism adaptation-like training in virtual reality (VRPA) in combination with functional neuroimaging. Three groups of healthy participants (N = 45) underwent VRPA (with a shift either to the left/right side, or with no shift), and performed functional magnetic resonance imaging (fMRI) sessions before and after training. To capture modulations in free-flowing, task-free brain activity, the fMRI sessions included resting-state and free-viewing of naturalistic videos. We found significant decreases in spontaneous functional connectivity between attentional and default mode (DMN)/fronto-parietal networks, only for the adaptation groups, more pronouncedly in the hemisphere contralateral to the induced shift. In addition, VRPA was found to bias visual responses to naturalistic videos: Following rightward adaptation, we found upregulation of visual response in an area in the parieto-occipital sulcus (POS) only in the right hemisphere. Notably, the extent of POS upregulation correlated with the size of the VRPA-induced after-effect measured in behavioral tests. This study demonstrates that a brief VRPA exposure can change large-scale cortical connectivity and correspondingly bias visual responses to naturalistic sensory inputs.

An evaluation of how connectopic mapping reveals visual field maps in V1

Abstract

Functional gradients, in which response properties change gradually across the cortical surface, have been proposed as a key organising principle of the brain. However, the presence of these gradients remains undetermined in many brain regions. Resting-state neuroimaging studies have suggested these gradients can be reconstructed from patterns of functional connectivity. Here we investigate the accuracy of these reconstructions and establish whether it is connectivity or the functional properties within a region that determine these “connectopic maps”. Different manifold learning techniques were used to recover visual field maps while participants were at rest or engaged in natural viewing. We benchmarked these reconstructions against maps measured by traditional visual field mapping. We report an initial exploratory experiment of a publicly available naturalistic imaging dataset, followed by a preregistered replication using larger resting-state and naturalistic imaging datasets from the Human Connectome Project. Connectopic mapping accurately predicted visual field maps in primary visual cortex, with better predictions for eccentricity than polar angle maps. Non-linear manifold learning methods outperformed simpler linear embeddings. We also found more accurate predictions during natural viewing compared to resting-state. Varying the source of the connectivity estimates had minimal impact on the connectopic maps, suggesting the key factor is the functional topography within a brain region. The application of these standardised methods for connectopic mapping will allow the discovery of functional gradients across the brain.

Protocol registration

The stage 1 protocol for this Registered Report was accepted in principle on 19 April 2022. The protocol, as accepted by the journal, can be found at 10.6084/m9.figshare.19771717.

Visuospatial coding as ubiquitous scaffolding for human cognition

For more than 100 years we have known that the visual field is mapped onto the surface of visual cortex, imposing an inherently spatial reference frame on visual information processing. Recent studies highlight visuospatial coding not only throughout visual cortex, but also brain areas not typically considered visual. Such widespread access to visuospatial coding raises important questions about its role in wider cognitive functioning. Here, we synthesise these recent developments and propose that visuospatial coding scaffolds human cognition by providing a reference frame through which neural computations interface with environmental statistics and task demands via perception-action loops.

Divisive normalization unifies disparate response signatures throughout the human visual hierarchy

A canonical neural computation is a mathematical operation applied by the brain in a wide variety of contexts and capable of explaining and unifying seemingly unrelated neural and perceptual phenomena. Here, we use a combination of state-of-the-art experiments (ultra-high-field functional MRI) and mathematical methods (population receptive field [pRF] modeling) to uniquely demonstrate the role of divisive normalization (DN) as the canonical neural computation underlying visuospatial responses throughout the human visual hierarchy. The DN pRF model provides a tool to investigate and interpret the computational processes underlying neural responses in human and animal recordings, but also in clinical and cognitive dimensions.

Neural processing is hypothesized to apply the same mathematical operations in a variety of contexts, implementing so-called canonical neural computations. Divisive normalization (DN) is considered a prime candidate for a canonical computation. Here, we propose a population receptive field (pRF) model based on DN and evaluate it using ultra-high-field functional MRI (fMRI). The DN model parsimoniously captures seemingly disparate response signatures with a single computation, superseding existing pRF models in both performance and biological plausibility. We observe systematic variations in specific DN model parameters across the visual hierarchy and show how they relate to differences in response modulation and visuospatial information integration. The DN model delivers a unifying framework for visuospatial responses throughout the human visual hierarchy and provides insights into its underlying information-encoding computations. These findings extend the role of DN as a canonical computation to neuronal populations throughout the human visual hierarchy.

Advances in resting state fMRI acquisitions for functional connectomics

Resting state functional magnetic resonance imaging (rs-fMRI) is based on spontaneous fluctuations in the blood oxygen level dependent (BOLD) signal, which occur simultaneously in different brain regions, without the subject performing an explicit task. The low-frequency oscillations of the rs-fMRI signal demonstrate an intrinsic spatiotemporal organization in the brain (brain networks) that may relate to the underlying neural activity. In this review article, we briefly describe the current acquisition techniques for rs-fMRI data, from the most common approaches for resting state acquisition strategies, to more recent investigations with dedicated hardware and ultra-high fields. Specific sequences that allow very fast acquisitions, or multiple echoes, are discussed next. We then consider how acquisition methods weighted towards specific parts of the BOLD signal, like the Cerebral Blood Flow (CBF) or Volume (CBV), can provide more spatially specific network information. These approaches are being developed alongside the commonly used BOLD-weighted acquisitions. Finally, specific applications of rs-fMRI to challenging regions such as the laminae in the neocortex, and the networks within the large areas of subcortical white matter regions are discussed. We finish the review with recommendations for acquisition strategies for a range of typical applications of resting state fMRI.

Neural Correlates of Subsequent Memory-Related Gaze Reinstatement

Mounting evidence linking gaze reinstatement-the recapitulation of encoding-related gaze patterns during retrieval-to behavioral measures of memory suggests that eye movements play an important role in mnemonic processing. Yet, the nature of the gaze scanpath, including its informational content and neural correlates, has remained in question. In this study, we examined eye movement and neural data from a recognition memory task to further elucidate the behavioral and neural bases of functional gaze reinstatement. Consistent with previous work, gaze reinstatement during retrieval of freely viewed scene images was greater than chance and predictive of recognition memory performance. Gaze reinstatement was also associated with viewing of informationally salient image regions at encoding, suggesting that scanpaths may encode and contain high-level scene content. At the brain level, gaze reinstatement was predicted by encoding-related activity in the occipital pole and BG, neural regions associated with visual processing and oculomotor control. Finally, cross-voxel brain pattern similarity analysis revealed overlapping subsequent memory and subsequent gaze reinstatement modulation effects in the parahippocampal place area and hippocampus, in addition to the occipital pole and BG. Together, these findings suggest that encoding-related activity in brain regions associated with scene processing, oculomotor control, and memory supports the formation, and subsequent recapitulation, of functional scanpaths. More broadly, these findings lend support to scanpath theory's assertion that eye movements both encode, and are themselves embedded in, mnemonic representations.

Voxel-to-voxel predictive models reveal unexpected structure in unexplained variance

Encoding models based on deep convolutional neural networks (DCNN) predict BOLD responses to natural scenes in the human visual system more accurately than many other currently available models. However, DCNN-based encoding models fail to predict a significant amount of variance in the activity of most voxels in all visual areas. This failure could reflect limitations in the data (e.g., a noise ceiling), or could reflect limitations of the DCNN as a model of computation in the brain. Understanding the source and structure of the unexplained variance could therefore provide helpful clues for improving models of brain computation. Here, we characterize the structure of the variance that DCNN-based encoding models cannot explain. Using a publicly available dataset of BOLD responses to natural scenes, we determined if the source of unexplained variance was shared across voxels, individual brains, retinotopic locations, and hierarchically distant visual brain areas. We answered these questions using voxel-to-voxel (vox2vox) models that predict activity in a target voxel given activity in a population of source voxels. We found that simple linear vox2vox models increased within-subject prediction accuracy over DCNN-based models for any pair of source/target visual areas, clearly demonstrating that the source of unexplained variance is widely shared within and across visual brain areas. However, vox2vox models were not more accurate than DCNN-based encoding models when source and target voxels came from different brains, demonstrating that the source of unexplained variance was not shared across brains. Importantly, control analyses demonstrated that the source of unexplained variance was not encoded in the mean activity of source voxels, or the activity of voxels in white matter. Interestingly, the weights of vox2vox models revealed preferential connection of target voxel activity to source voxels with adjacent receptive fields, even when source and target voxels were in different functional brain areas. Finally, we found that the prediction accuracy of the vox2vox models decayed with hierarchical distance between the source and target voxels but showed detailed patterns of dependence on hierarchical relationships that we did not observe in DCNNs. Given these results, we argue that the structured variance unexplained by DCNN-based encoding models is unlikely to be entirely caused by non-neural artifacts (e.g., spatially correlated measurement noise) or a failure of DCNNs to approximate the features encoded in brain activity; rather, our results point to a need for brain models that provide both mechanistic and computational explanations for structured ongoing activity in the brain. Keywords: fMRI, encoding models, deep neural networks, functional connectivity.

Self-reference Network-Related Interactions During the Process of Cognitive Impairment in the Early Stages of Alzheimer's Disease

Background: Normal establishment of cognition occurs after forming a sensation to stimuli from internal or external cues, in which self-reference processing may be partially involved. However, self-reference processing has been less studied in the Alzheimer’s disease (AD) field within the self-reference network (SRN) and has instead been investigated within the default-mode network (DMN). Differences between these networks have been proven in the last decade, while ultra-early diagnoses have increased. Therefore, investigation of the altered pattern of SRN is significantly important, especially in the early stages of AD.

Methods: A total of 65 individuals, including 43 with mild cognitive impairment (MCI) and 22 cognitively normal individuals, participated in this study. The SRN, dorsal attention network (DAN), and salience network (SN) were constructed with resting-state functional magnetic resonance imaging (fMRI), and voxel-based analysis of variance (ANOVA) was used to explore significant regions of network interactions. Finally, the correlation between the network interactions and clinical characteristics was analyzed.

Results: We discovered four interactions among the three networks, with the SRN showing different distributions in the left and right hemispheres from the DAN and SN and modulated interactions between them. Group differences in the interactions that were impaired in MCI patients indicated that the degree of damage was most severe in the SRN, least severe in the SN, and intermediate in the DAN. The two SRN-related interactions showed positive effects on the executive and memory performances of MCI patients with no overlap with the clinical assessments performed in this study.

Conclusion: This study is the first and primary evidence of SRN interactions related to MCI patients’ functional performance. The influence of the SRN in the ultra-early stages of AD is nonnegligible. There are still many unknowns regarding the contribution of the SRN in AD progression, and we strongly recommend future research in this area.

Representation of Contralateral Visual Space in the Human Hippocampus

The initial encoding of visual information primarily from the contralateral visual field is a fundamental organizing principle of the primate visual system. Recently, the presence of such retinotopic sensitivity has been shown to extend well beyond early visual cortex to regions not historically considered retinotopically sensitive. In particular, human scene-selective regions in parahippocampal and medial parietal cortex exhibit prominent biases for the contralateral visual field. Here, we used fMRI to test the hypothesis that the human hippocampus, which is thought to be anatomically connected with these scene-selective regions, would also exhibit a biased representation of contralateral visual space. First, population receptive field (pRF) mapping with scene stimuli revealed strong biases for the contralateral visual field in bilateral hippocampus. Second, the distribution of retinotopic sensitivity suggested a more prominent representation in anterior medial portions of the hippocampus. Finally, the contralateral bias was confirmed in independent data taken from the Human Connectome Project (HCP) initiative. The presence of contralateral biases in the hippocampus, a structure considered by many as the apex of the visual hierarchy, highlights the truly pervasive influence of retinotopy. Moreover, this finding has important implications for understanding how visual information relates to the allocentric global spatial representations known to be encoded therein.SIGNIFICANCE STATEMENT Retinotopic encoding of visual information is an organizing principle of visual cortex. Recent work demonstrates this sensitivity in structures far beyond early visual cortex, including those anatomically connected to the hippocampus. Here, using population receptive field (pRF) modeling in two independent sets of data we demonstrate a consistent bias for the contralateral visual field in bilateral hippocampus. Such a bias highlights the truly pervasive influence of retinotopy, with important implications for understanding how the presence of retinotopy relates to more allocentric spatial representations.

The thermodynamics of cognition: A mathematical treatment

There is a general expectation that the laws of classical physics must apply to biology, particularly the neural system. The evoked cycle represents the brain's energy/information exchange with the physical environment through stimulus. Therefore, the thermodynamics of emotions might elucidate the neurological origin of intellectual evolution, and explain the psychological and health consequences of positive and negative emotional states based on their energy profiles. We utilized the Carnot cycle and Landauer's principle to analyze the energetic consequences of the brain's resting and evoked states during and after various cognitive states. Namely, positive emotional states can be represented by the reversed Carnot cycle, whereas negative emotional reactions trigger the Carnot cycle. The two conditions have contrasting energetic and entropic aftereffects with consequences for mental energy. The mathematics of the Carnot and reversed Carnot cycles, which can explain recent findings in human psychology, might be constructive in the scientific endeavor in turning psychology into hard science.