Compressive Temporal Summation in Human Visual Cortex

Spatial Frequency Maps in Human Visual Cortex: A Replication and Extension

In a step toward developing a model of human primary visual cortex, a recent study introduced a model of spatial frequency tuning in V1 (Broderick, Simoncelli, & Winawer, 2022). The model is compact, using just 9 parameters to predict BOLD response amplitude for locations across all of V1 as a function of stimulus orientation and spatial frequency. Here we replicated this analysis in a new dataset, the 'nsdsynthetic' supplement to the Natural Scenes Dataset (Allen et al., 2022), to assess generalization of model parameters. Furthermore, we extended the analyses to extrastriate maps V2 and V3. For each retinotopic map in the 8 NSD subjects, we fit the 9-parameter model. Despite many experimental differences between NSD and the original study, including stimulus size, experimental design, and MR field strength, there was good agreement in most model parameters. The dependence of preferred spatial frequency on eccentricity in V1 was similar between NSD and Broderick et al. Moreover, the effect of absolute stimulus orientation on spatial frequency maps was similar: higher preferred spatial frequency for horizontal and cardinal orientations compared to vertical and oblique orientations in both studies. The extension to extrastriate maps revealed that the biggest change in tuning between maps was in bandwidth: the bandwidth in spatial frequency tuning increased by 70% from V1 to V2 and 100% from V1 to V3, paralleling known increases in receptive field size. Together, the results show robust reproducibility and bring us closer to a systematic characterization of spatial encoding in the human visual system.

Readiness for Perception and Action: Towards a More Mechanistic Understanding of Phasic Alertness

Human survival requires prompt perception and action to address relevant events in the environment. For this, the brain has evolved a system that uses warning stimuli to elicit phasic alertness, a state readying the brain for upcoming perception and action. Although a wealth of empirical evidence revealed how phasic alertness improves a wide range of perceptual and cognitive processing, it is still unclear by what cognitive mechanisms this is achieved. Here, we identify key problems that have to be solved for this to be possible and delineate concrete ways to achieve this. Specifically, we discover I) how to establish phasic alertness as a cognitive state of readiness for perception and action, II) how it can affect cognition online or offline, III) how it could be triggered internally without a warning, and IV) to what degrees it relied on bottom-up processing, or top-down temporal or stimulus expectations and the current task. As a result, the discussion provides us with a research program yielding the theoretical and empirical basis for mechanistic and computational models of phasic alertness and its neurophysiological underpinnings.

Improving the reliability and accuracy of population receptive field measures using a logarithmically warped stimulus

The population receptive field (pRF) method, which measures the region in visual space that elicits a blood-oxygen-level-dependent (BOLD) signal in a voxel in retinotopic cortex, is a powerful tool for investigating the functional organization of human visual cortex with fMRI (Dumoulin & Wandell, 2008). However, recent work has shown that pRF estimates for early retinotopic visual areas can be biased and unreliable, especially for voxels representing the fovea. Here, we show that a log-bar stimulus that is logarithmically warped along the eccentricity dimension produces more reliable estimates of pRF size and location than the traditional moving bar stimulus. The log-bar stimulus was better able to identify pRFs near the foveal representation, and pRFs were smaller in size, consistent with simulation estimates of receptive field sizes in the fovea.

The neural link between stimulus duration and spatial location in the human visual hierarchy

Integrating spatial and temporal information is essential for our sensory experience. While psychophysical evidence suggests spatial dependencies in duration perception, few studies have directly tested the neural link between temporal and spatial processing. Using ultra-high-field functional MRI and neuronal-based modeling, we investigated how and where the processing and the representation of a visual stimulus duration is linked to that of its spatial location. Our results show a transition in duration coding: from monotonic and spatially-dependent in early visual cortex to unimodal and spatially-invariant in frontal cortex. Along the dorsal visual stream, particularly in the intraparietal sulcus (IPS), neuronal populations show common selective responses to both spatial and temporal information. In the IPS, spatial and temporal topographic organizations are also linked, although duration maps are smaller, less clustered, and more variable across participants. These findings help identify the mechanisms underlying human perception of visual duration and characterize the functional link between time and space processing, highlighting the importance of their interactions in shaping brain responses.

Revealing the Causes of Dyslexia through a Differential Diagnosis, a Short-Term Effective Treatment and an Appropriate Conceptual Framework

Various different impairments and their interactions can cause reading problems referred to as "dyslexia". Since reading requires the interaction of many abilities, the impairment of each of these abilities can result in dyslexia. Therefore, the diagnosis must differentiate various kinds of dyslexia. The diagnosis of a certain kind of dyslexia cannot be delimited to the investigation and description of symptoms but must also include the investigation of the causes of each kind of dyslexia. For this purpose, a scientifically unequivocal concept of causation and appropriate methods are needed to distinguish them from co-existing impairments that have no causal influence on reading performance. The results of applying these methods cannot be adequately accounted for by a non-scientific, intuitive understanding of necessary and sufficient conditions and causation. The methods suitable for revealing the causes of dyslexia are described in detail, and the results of applying these methods in experiments, in which 356 children with developmental dyslexia participated, are reviewed. Since the concepts of "necessary" and "sufficient" conditions and "causation" proposed in the philosophy of science are not suitable for describing causes of dyslexia and their interaction, they are replaced by a more detailed, experimentally based conceptual framework that provides an accurate description of the conditions required for correct reading and the causes of dyslexia.

Rethinking simultaneous suppression in visual cortex via compressive spatiotemporal population receptive fields

When multiple visual stimuli are presented simultaneously in the receptive field, the neural response is suppressed compared to presenting the same stimuli sequentially. The prevailing hypothesis suggests that this suppression is due to competition among multiple stimuli for limited resources within receptive fields, governed by task demands. However, it is unknown how stimulus-driven computations may give rise to simultaneous suppression. Using fMRI, we find simultaneous suppression in single voxels, which varies with both stimulus size and timing, and progressively increases up the visual hierarchy. Using population receptive field (pRF) models, we find that compressive spatiotemporal summation rather than compressive spatial summation predicts simultaneous suppression, and that increased simultaneous suppression is linked to larger pRF sizes and stronger compressive nonlinearities. These results necessitate a rethinking of simultaneous suppression as the outcome of stimulus-driven compressive spatiotemporal computations within pRFs, and open new opportunities to study visual processing capacity across space and time.

Improving the Reliability and Accuracy of Population Receptive Field Measures Using a Logarithmically Warped Stimulus

The population receptive field method, which measures the region in visual space that elicits a BOLD signal in a voxel in retinotopic cortex, is a powerful tool for investigating the functional organization of human visual cortex with fMRI (Dumoulin & Wandell, 2008). However, recent work has shown that population receptive field (pRF) estimates for early retinotopic visual areas can be biased and unreliable, especially for voxels representing the fovea. Here, we show that a 'log-bar' stimulus that is logarithmically warped along the eccentricity dimension produces more reliable estimates of pRF size and location than the traditional moving bar stimulus. The log-bar stimulus was better able to identify pRFs near the foveal representation, and pRFs were smaller in size, consistent with simulation estimates of receptive field sizes in the fovea.

Disparate nonlinear neural dynamics measured with different techniques in macaque and human V1

Diverse neuro-imaging techniques measure different aspects of neural responses with distinct spatial and temporal resolutions. Relating measured neural responses across different methods has been challenging. Here, we take a step towards overcoming this challenge, by comparing the nonlinearity of neural dynamics measured across methods. We used widefield voltage-sensitive dye imaging (VSDI) to measure neural population responses in macaque V1 to visual stimuli with a wide range of temporal waveforms. We found that stimulus-evoked VSDI responses are surprisingly near-additive in time. These results are qualitatively different from the strong sub-additive dynamics previously measured using fMRI and electrocorticography (ECoG) in human visual cortex with a similar set of stimuli. To test whether this discrepancy is specific to VSDI-a signal dominated by subthreshold neural activity, we repeated our measurements using widefield imaging of a genetically encoded calcium indicator (GcaMP6f)-a signal dominated by spiking activity, and found that GCaMP signals in macaque V1 are also near-additive. Therefore, the discrepancies in the extent of sub-additivity between the macaque and the human measurements are unlikely due to differences between sub- and supra-threshold neural responses. Finally, we use a simple yet flexible delayed normalization model to capture these different dynamics across measurements (with different model parameters). The model can potentially generalize to a broader set of stimuli, which aligns with previous suggestion that dynamic gain-control is a canonical computation contributing to neural processing in the brain.

Temporal dynamics of short-term neural adaptation across human visual cortex

Neural responses in visual cortex adapt to prolonged and repeated stimuli. While adaptation occurs across the visual cortex, it is unclear how adaptation patterns and computational mechanisms differ across the visual hierarchy. Here we characterize two signatures of short-term neural adaptation in time-varying intracranial electroencephalography (iEEG) data collected while participants viewed naturalistic image categories varying in duration and repetition interval. Ventral- and lateral-occipitotemporal cortex exhibit slower and prolonged adaptation to single stimuli and slower recovery from adaptation to repeated stimuli compared to V1-V3. For category-selective electrodes, recovery from adaptation is slower for preferred than non-preferred stimuli. To model neural adaptation we augment our delayed divisive normalization (DN) model by scaling the input strength as a function of stimulus category, enabling the model to accurately predict neural responses across multiple image categories. The model fits suggest that differences in adaptation patterns arise from slower normalization dynamics in higher visual areas interacting with differences in input strength resulting from category selectivity. Our results reveal systematic differences in temporal adaptation of neural population responses between lower and higher visual brain areas and show that a single computational model of history-dependent normalization dynamics, fit with area-specific parameters, accounts for these differences.

Feature-invariant processing of spatial segregation based on temporal asynchrony

Temporal asynchrony is a cue for the perceptual segregation of spatial regions. Past research found attribute invariance of this phenomenon such that asynchrony induces perceptual segmentation regardless of the changing attribute type, and it does so even when asynchrony occurs between different attributes. To test the generality of this finding and obtain insights into the underlying computational mechanism, we compared the segmentation performance for changes in luminance, color, motion direction, and their combinations. Our task was to detect the target quadrant in which a periodic alternation in attribute was phase-delayed compared to the remaining quadrants. When stimulus elements made a square-wave attribute change, target detection was not clearly attribute invariant, being more difficult for motion direction change than for luminance or color changes and nearly impossible for the combination of motion direction and luminance or color. We suspect that waveform mismatch might cause anomalous behavior of motion direction since a square-wave change in motion direction is a triangular-wave change in the spatial phase (i.e., a second-order change in the direction of the spatial phase change). In agreement with this idea, we found that the segregation performance was strongly affected by the waveform type (square wave, triangular wave, or their combination), and when this factor was controlled, the performance was nearly, though not perfectly, invariant against attribute type. The results were discussed with a model in which different visual attributes share a common asynchrony-based segmentation mechanism.

Rethinking simultaneous suppression in visual cortex via compressive spatiotemporal population receptive fields

When multiple visual stimuli are presented simultaneously in the receptive field, the neural response is suppressed compared to presenting the same stimuli sequentially. The prevailing hypothesis suggests that this suppression is due to competition among multiple stimuli for limited resources within receptive fields, governed by task demands. However, it is unknown how stimulus-driven computations may give rise to simultaneous suppression. Using fMRI, we find simultaneous suppression in single voxels, which varies with both stimulus size and timing, and progressively increases up the visual hierarchy. Using population receptive field (pRF) models, we find that compressive spatiotemporal summation rather than compressive spatial summation predicts simultaneous suppression, and that increased simultaneous suppression is linked to larger pRF sizes and stronger compressive nonlinearities. These results necessitate a rethinking of simultaneous suppression as the outcome of stimulus-driven compressive spatiotemporal computations within pRFs, and open new opportunities to study visual processing capacity across space and time.

Intracortical recordings reveal vision-to-action cortical gradients driving human exogenous attention

Exogenous attention, the process that makes external salient stimuli pop-out of a visual scene, is essential for survival. How attention-capturing events modulate human brain processing remains unclear. Here we show how the psychological construct of exogenous attention gradually emerges over large-scale gradients in the human cortex, by analyzing activity from 1,403 intracortical contacts implanted in 28 individuals, while they performed an exogenous attention task. The timing, location and task-relevance of attentional events defined a spatiotemporal gradient of three neural clusters, which mapped onto cortical gradients and presented a hierarchy of timescales. Visual attributes modulated neural activity at one end of the gradient, while at the other end it reflected the upcoming response timing, with attentional effects occurring at the intersection of visual and response signals. These findings challenge multi-step models of attention, and suggest that frontoparietal networks, which process sequential stimuli as separate events sharing the same location, drive exogenous attention phenomena such as inhibition of return.

Electrophysiological correlates of temporal numerosity adaptation

Introduction

Much research has revealed the human visual system is capable to estimate numerical quantities, rapidly and reliably, in both the spatial and the temporal domain. This ability is highly susceptible to short-term plastic phenomena related to previous exposure to visual numerical information (i.e., adaptation). However, while determinants of spatial numerosity adaptation have been widely investigated, little is known about the neural underpinnings of short-term plastic phenomena related to the encoding of temporal numerical information. In the present study we investigated the electrophysiological correlates of temporal numerosity adaptation.

Methods

Participants were asked to estimate the numerosity of a test sequence of flashes after being exposed to either a high or low numerous adapting sequence. Behavioral results confirmed the expected underestimation of test stimulus when this was preceded by a high numerous sequence as compared to when preceded by a low numerous sequence.

Results

Electrophysiological data revealed that this behavior was tightly linked to the amplitude of the steady-state visual evoked (ssVEP) response elicited by the test stimulus. When preceded by a high numerous sequence, the test stimulus elicited larger ssVEP responses as compared to when preceded by a low numerous sequence with this pattern being robustly correlated with behavior. Finally, topographical maps showed that this difference was mostly evident across two antero-posterior distributed clusters of electrodes and correlated with changes in functional connectivity.

Discussion

Taken together, our results suggest that visual plastic phenomena related to the encoding of temporal numerosity information reflect changes in rhythmic evoked activity that are likely related to long range communications between distinct brain regions.

Temporal dynamics of short-term neural adaptation across human visual cortex

Neural responses in visual cortex adapt to prolonged and repeated stimuli. While adaptation occurs across the visual cortex, it is unclear how adaptation patterns and computational mechanisms differ across the visual hierarchy. Here we characterize two signatures of short-term neural adaptation in time-varying intracranial electroencephalography (iEEG) data collected while participants viewed naturalistic image categories varying in duration and repetition interval. Ventral- and lateral-occipitotemporal cortex exhibit slower and prolonged adaptation to single stimuli and slower recovery from adaptation to repeated stimuli compared to V1-V3. For category-selective electrodes, recovery from adaptation is slower for preferred than non-preferred stimuli. To model neural adaptation we augment our delayed divisive normalization (DN) model by scaling the input strength as a function of stimulus category, enabling the model to accurately predict neural responses across multiple image categories. The model fits suggest that differences in adaptation patterns arise from slower normalization dynamics in higher visual areas interacting with differences in input strength resulting from category selectivity. Our results reveal systematic differences in temporal adaptation of neural population responses across the human visual hierarchy and show that a single computational model of history-dependent normalization dynamics, fit with area-specific parameters, accounts for these differences.

Characterizing Spatiotemporal Population Receptive Fields in Human Visual Cortex with fMRI

The use of fMRI and computational modeling has advanced understanding of spatial characteristics of population receptive fields (pRFs) in human visual cortex. However, we know relatively little about the spatiotemporal characteristics of pRFs because neurons' temporal properties are one to two orders of magnitude faster than fMRI BOLD responses. Here, we developed an image-computable framework to estimate spatiotemporal pRFs from fMRI data. First, we developed a simulation software that predicts fMRI responses to a time-varying visual input given a spatiotemporal pRF model and solves the model parameters. The simulator revealed that ground-truth spatiotemporal parameters can be accurately recovered at the millisecond resolution from synthesized fMRI responses. Then, using fMRI and a novel stimulus paradigm, we mapped spatiotemporal pRFs in individual voxels across human visual cortex in 10 participants (both females and males). We find that a compressive spatiotemporal (CST) pRF model better explains fMRI responses than a conventional spatial pRF model across visual areas spanning the dorsal, lateral, and ventral streams. Further, we find three organizational principles of spatiotemporal pRFs: (1) from early to later areas within a visual stream, spatial and temporal windows of pRFs progressively increase in size and show greater compressive nonlinearities, (2) later visual areas show diverging spatial and temporal windows across streams, and (3) within early visual areas (V1-V3), both spatial and temporal windows systematically increase with eccentricity. Together, this computational framework and empirical results open exciting new possibilities for modeling and measuring fine-grained spatiotemporal dynamics of neural responses using fMRI.

Computational model links normalization to chemoarchitecture in the human visual system

A goal of cognitive neuroscience is to provide computational accounts of brain function. Canonical computations-mathematical operations used by the brain in many contexts-fulfill broad information-processing needs by varying their algorithmic parameters. A key question concerns the identification of biological substrates for these computations and their algorithms. Chemoarchitecture-the spatial distribution of neurotransmitter receptor densities-shapes brain function. Here, we propose that local variations in specific receptor densities implement algorithmic modulations of canonical computations. To test this hypothesis, we combine mathematical modeling of brain responses with chemoarchitecture data. We compare parameters of divisive normalization obtained from 7-tesla functional magnetic resonance imaging with receptor density maps obtained from positron emission tomography. We find evidence that serotonin and γ-aminobutyric acid receptor densities are the biological substrate for algorithmic modulations of divisive normalization in the human visual system. Our model links computational and biological levels of vision, explaining how canonical computations allow the brain to fulfill broad information-processing needs.

Effective connectivity in a duration selective cortico-cerebellar network

How the human brain represents millisecond unit of time is far from clear. A recent neuroimaging study revealed the existence in the human premotor cortex of a topographic representation of time i.e., neuronal units selectively responsive to specific durations and topographically organized on the cortical surface. By using high resolution functional Magnetic Resonance Images here, we go beyond this previous work, showing duration preferences across a wide network of cortical and subcortical brain areas: from cerebellum to primary visual, parietal, premotor and prefrontal cortices. Most importantly, we identify the effective connectivity structure between these different brain areas and their duration selective neural units. The results highlight the role of the cerebellum as the network hub and that of medial premotor cortex as the final stage of duration recognition. Interestingly, when a specific duration is presented, only the communication strength between the units selective to that specific duration and to the neighboring durations is affected. These findings link for the first time, duration preferences within single brain region with connectivity dynamics between regions, suggesting a communication mode that is partially duration specific.

Dyslexia Due to Visual Impairments

Reading involves many different abilities that are necessary or sufficient conditions for fluent and flawless reading. The absence of one necessary or of all sufficient conditions is a cause of dyslexia. The present study investigates whether too short fixation times and an impaired ability to recognize a string of letters simultaneously are causes of dyslexia. The frequency and types of reading mistakes were investigated in a tachistoscopic pseudoword experiment with 100 children with dyslexia to test the impact of too short fixation times and the attempts of children with dyslexia to recognize more letters simultaneously than they can when reading pseudowords. The experiment demonstrates that all types of reading mistakes disappear when the fixation time increases and/or the number of letters that the children try to recognize simultaneously is reduced. The results cannot be interpreted as being due to altered visual crowding, impaired attention, or impaired phonological awareness, but can be regarded as an effect of impaired temporal summation and a dysfunction in the ventral stream of the visual system.

The effect of familiarity on behavioral oscillations in face perception

Studies on behavioral oscillations demonstrate that visual sensitivity fluctuates over time and visual processing varies periodically, mirroring neural oscillations at the same frequencies. Do these behavioral oscillations reflect fixed and relatively automatic sensory sampling, or top-down processes such as attention or predictive coding? To disentangle these theories, the current study used a dual-target rapid serial visual presentation paradigm, where participants indicated the gender of a face target embedded in streams of distractors presented at 30 Hz. On critical trials, two identical targets were presented with varied stimulus onset asynchrony from 200 to 833 ms. The target was either familiar or unfamiliar faces, divided into different blocks. We found a 4.6 Hz phase-coherent fluctuation in gender discrimination performance across both trial types, consistent with previous reports. In addition, however, we found an effect at the alpha frequency, with behavioral oscillations in the familiar blocks characterized by a faster high-alpha peak than for the unfamiliar face blocks. These results are consistent with the combination of both a relatively stable modulation in the theta band and faster modulation of the alpha oscillations. Therefore, the overall pattern of perceptual sampling in visual perception may depend, at least in part, on task demands. PROTOCOL REGISTRATION: The stage 1 protocol for this Registered Report was accepted in principle on 16/08/2022. The protocol, as accepted by the journal, can be found at: https://doi.org/10.17605/OSF.IO/A98UF .

Characterizing spatiotemporal population receptive fields in human visual cortex with fMRI

The use of fMRI and computational modeling has advanced understanding of spatial characteristics of population receptive fields (pRFs) in human visual cortex. However, we know relatively little about the spatiotemporal characteristics of pRFs because neurons' temporal properties are one to two orders of magnitude faster than fMRI BOLD responses. Here, we developed an image-computable framework to estimate spatiotemporal pRFs from fMRI data. First, we developed a simulation software that predicts fMRI responses to a time varying visual input given a spatiotemporal pRF model and solves the model parameters. The simulator revealed that ground-truth spatiotemporal parameters can be accurately recovered at the millisecond resolution from synthesized fMRI responses. Then, using fMRI and a novel stimulus paradigm, we mapped spatiotemporal pRFs in individual voxels across human visual cortex in 10 participants. We find that a compressive spatiotemporal (CST) pRF model better explains fMRI responses than a conventional spatial pRF model across visual areas spanning the dorsal, lateral, and ventral streams. Further, we find three organizational principles of spatiotemporal pRFs: (i) from early to later areas within a visual stream, spatial and temporal integration windows of pRFs progressively increase in size and show greater compressive nonlinearities, (ii) later visual areas show diverging spatial and temporal integration windows across streams, and (iii) within early visual areas (V1-V3), both spatial and temporal integration windows systematically increase with eccentricity. Together, this computational framework and empirical results open exciting new possibilities for modeling and measuring fine-grained spatiotemporal dynamics of neural responses in the human brain using fMRI.

Neural timescales reflect behavioral demands in freely moving rhesus macaques

Previous work has demonstrated remarkably reproducible and consistent hierarchies of neural timescales across cortical areas at rest. The question arises how such stable hierarchies give rise to adaptive behavior that requires flexible adjustment of temporal coding and integration demands. Potentially, this previously found lack of variability in the hierarchical organization of neural timescales could be a reflection of the structure of the laboratory contexts in which they were measured. Indeed, computational work demonstrates the existence of multiple temporal hierarchies within the same anatomical network when the input structure is altered. We posit that unconstrained behavioral environments where relatively little temporal demands are imposed from the experimenter are an ideal test bed to address the question of whether the hierarchical organization and the magnitude of neural timescales reflect ongoing behavioral demands. To tackle this question, we measured timescales of local field potential activity while rhesus macaques were foraging freely in a large open space. We find a hierarchy of neural timescales that is unique to this foraging environment. Importantly, although the magnitude of neural timescales generally expanded with task engagement, the brain areas' relative position in the hierarchy was stable across the recording sessions. Notably, the magnitude of neural timescales monotonically expanded with task engagement across a relatively long temporal scale spanning the duration of the recording session. Over shorter temporal scales, the magnitude of neural timescales changed dynamically around foraging events. Moreover, the change in the magnitude of neural timescales contained functionally relevant information, differentiating between seemingly similar events in terms of motor demands and associated reward. That is, the patterns of change were associated with the cognitive and behavioral meaning of these events. Finally, we demonstrated that brain areas were differentially affected by these behavioral demands - i.e., the expansion of neural timescales was not the same across all areas. Together, these results demonstrate that the observed hierarchy of neural timescales is context-dependent and that changes in the magnitude of neural timescales are closely related to overall task engagement and behavioral demands.

Dyslexia: Causes and Concomitant Impairments

In recent decades, theories have been presented to explain the nature of dyslexia, but the causes of dyslexia remained unclear. Although the investigation of the causes of dyslexia presupposes a clear understanding of the concept of cause, such an understanding is missing. The present paper proposes the absence of at least one necessary condition or the absence of all sufficient conditions as causes for impaired reading. The causes of impaired reading include: an incorrect fixation location, too short a fixation time, the attempt to recognize too many letters simultaneously, too large saccade amplitudes, and too short verbal reaction times. It is assumed that a longer required fixation time in dyslexic readers results from a functional impairment of areas V1, V2, and V3 that require more time to complete temporal summation. These areas and areas that receive input from them, such as the fusiform gyrus, are assumed to be impaired in their ability to simultaneously process a string of letters. When these impairments are compensated by a new reading strategy, reading ability improves immediately.

Stress disrupts insight-driven mnemonic reconfiguration in the medial temporal lobe

Memories are not stored in isolation. Insight into the relationship of initially unrelated events may trigger a flexible reconfiguration of the mnemonic representation of these events. Such representational changes allow the integration of events into coherent episodes and help to build up-to-date-models of the world around us. This process is, however, frequently impaired in stress-related mental disorders resulting in symptoms such as fragmented memories in PTSD. Here, we combined a real life-like narrative-insight task, in which participants learned how initially separate events are linked, with fMRI-based representational similarity analysis to test if and how acute stress interferes with the insight-driven reconfiguration of memories. Our results showed that stress reduced the activity of medial temporal and prefrontal areas when participants gained insight into the link between events. Moreover, stress abolished the insight-related increase in representational dissimilarity for linked events in the anterior part of the hippocampus as well as its association with measures of subsequent memory that we observed in non-stressed controls. However, memory performance, as assessed in a forced-choice recognition test, was even enhanced in the stress group. Our findings suggest that acute stress impedes the neural integration of events into coherent episodes but promotes long-term memory for these integrated narratives and may thus have implications for understanding memory distortions in stress-related mental disorders.

Evidence suggesting common mechanisms underlie contralateral and ipsilateral negative BOLD responses in human visual cortex

The task-evoked positive BOLD response (PBR) to a unilateral visual hemi-field stimulation is often accompanied by robust and sustained contralateral as well as ipsilateral negative BOLD responses (NBRs) in the visual cortex. The signal characteristics and the neural and/or vascular mechanisms that underlie these two types of NBRs are not completely understood. In this paper, we investigated the properties of these two types of NBRs. We first demonstrated the linearity of both NBRs with respect to stimulus duration. Next, we showed that the hemodynamic response functions (HRFs) of the two NBRs were similar to each other, but significantly different from that of the PBR. Moreover, the subject-wise expressions of the two NBRs were tightly coupled to the degree that the correlation between the two NBRs was significantly higher than the correlation between each NBR and the PBR. However, the activation patterns of the two NBRs did not show a high level of interhemispheric spatial similarity, and the functional connectivity between them was not different than the interhemispheric functional connectivity between the NBRs and PBR. Finally, while attention did modulate both NBRs, the attention-related changes in their HRFs were similar. Our findings suggest that the two NBRs might be generated through common neural and/or vascular mechanisms involving distal/deep brain regions that project to the two hemispheres.

Visual timing-tuned responses in human association cortices and response dynamics in early visual cortex

Quantifying the timing (duration and frequency) of brief visual events is vital to human perception, multisensory integration and action planning. Tuned neural responses to visual event timing have been found in association cortices, in areas implicated in these processes. Here we ask how these timing-tuned responses are related to the responses of early visual cortex, which monotonically increase with event duration and frequency. Using 7-Tesla functional magnetic resonance imaging and neural model-based analyses, we find a gradual transition from monotonically increasing to timing-tuned neural responses beginning in the medial temporal area (MT/V5). Therefore, across successive stages of visual processing, timing-tuned response components gradually become dominant over inherent sensory response modulation by event timing. This additional timing-tuned response component is independent of retinotopic location. We propose that this hierarchical emergence of timing-tuned responses from sensory processing areas quantifies sensory event timing while abstracting temporal representations from spatial properties of their inputs.

Early visual cortical responses increase with event duration and frequency, while later timing-tuned responses quantify event timing. Here, the authors show timing tuning gradually emerges up the visual hierarchy, and separates temporal and spatial event features.

Trial-by-trial predictions of subjective time from human brain activity

Human experience of time exhibits systematic, context-dependent deviations from clock time; for example, time is experienced differently at work than on holiday. Here we test the proposal that differences from clock time in subjective experience of time arise because time estimates are constructed by accumulating the same quantity that guides perception: salient events. Healthy human participants watched naturalistic, silent videos of up to 24 seconds in duration and estimated their duration while fMRI was acquired. We were able to reconstruct trial-by-trial biases in participants’ duration reports, which reflect subjective experience of duration, purely from salient events in their visual cortex BOLD activity. By contrast, salient events in neither of two control regions–auditory and somatosensory cortex–were predictive of duration biases. These results held despite being able to (trivially) predict clock time from all three brain areas. Our results reveal that the information arising during perceptual processing of a dynamic environment provides a sufficient basis for reconstructing human subjective time duration.

Our perception of time isn’t like a clock; it varies depending on other aspects of experience, such as what we see and hear in that moment. Previous studies have shown that differences in simple features, such as an image being larger or smaller, or brighter or dimmer, can change how we perceive time for those experiences. But in everyday life, the properties of these simple features can change frequently, presenting a challenge to understanding real-world time perception based on simple lab experiments. To overcome this problem, we developed a computational model of human time perception based on tracking changes in neural activity across brain regions involved in sensory processing (using non-invasive brain imaging). By measuring changes in brain activity patterns across these regions, our approach accommodates the different and changing feature combinations present in natural scenarios, such as walking on a busy street. Our model reproduces people’s duration reports for natural videos (up to almost half a minute long) and, most importantly, predicts whether a person reports a scene as relatively shorter or longer–the biases in time perception that reflect how natural experience of time deviates from clock time.

Auditory timing-tuned neural responses in the human auditory cortices

Perception of sub-second auditory event timing supports multisensory integration, and speech and music perception and production. Neural populations tuned for the timing (duration and rate) of visual events were recently described in several human extrastriate visual areas. Here we ask whether the brain also contains neural populations tuned for auditory event timing, and whether these are shared with visual timing. Using 7T fMRI, we measured responses to white noise bursts of changing duration and rate. We analyzed these responses using neural response models describing different parametric relationships between event timing and neural response amplitude. This revealed auditory timing-tuned responses in the primary auditory cortex, and auditory association areas of the belt, parabelt and premotor cortex. While these areas also showed tonotopic tuning for auditory pitch, pitch and timing preferences were not consistently correlated. Auditory timing-tuned response functions differed between these areas, though without clear hierarchical integration of responses. The similarity of auditory and visual timing tuned responses, together with the lack of overlap between the areas showing these responses for each modality, suggests modality-specific responses to event timing are computed similarly but from different sensory inputs, and then transformed differently to suit the needs of each modality.

Individual resting-state alpha peak frequency and within-trial changes in alpha peak frequency both predict visual dual-pulse segregation performance

Although sensory input is continuous, information must be combined over time to guide action and cognition, leading to the proposal of temporal sampling windows. A number of studies have suggested that a 10-Hz sampling window might be involved in the "frame rate" of visual processing. To investigate this, we tested the ability of participants to localize and enumerate 1 or 2 visual flashes presented either at near-threshold or full-contrast intensities, while recording magnetoencephalography. The inter-stimulus interval (ISI) between the 2 flashes was varied across trials. Performance in distinguishing between 1 and 2 flashes was linked to the alpha frequency, both at the individual level and trial-by-trial. Participants with a higher resting-state alpha peak frequency showed the greatest improvement in performance as a function of ISI within a 100-ms time window, while those with slower alpha improved more when ISI exceeded 100 ms. On each trial, correct enumeration (1 vs. 2) performance was paired with faster pre-stimulus instantaneous alpha frequency. Our results suggest that visual sampling/processing speed, linked to peak alpha frequency, is both an individual trait and can vary in a state-dependent manner.

A dynamic normalization model of temporal attention

Vision is dynamic, handling a continuously changing stream of input, yet most models of visual attention are static. Here, we develop a dynamic normalization model of visual temporal attention and constrain it with new psychophysical human data. We manipulated temporal attention-the prioritization of visual information at specific points in time-to a sequence of two stimuli separated by a variable time interval. Voluntary temporal attention improved perceptual sensitivity only over a specific interval range. To explain these data, we modelled voluntary and involuntary attentional gain dynamics. Voluntary gain enhancement took the form of a limited resource over short time intervals, which recovered over time. Taken together, our theoretical and experimental results formalize and generalize the idea of limited attentional resources across space at a single moment to limited resources across time at a single location.

A population receptive field model of the magnetoencephalography response

Computational models which predict the neurophysiological response from experimental stimuli have played an important role in human neuroimaging. One type of computational model, the population receptive field (pRF), has been used to describe cortical responses at the millimeter scale using functional magnetic resonance imaging (fMRI) and electrocorticography (ECoG). However, pRF models are not widely used for non-invasive electromagnetic field measurements (EEG/MEG), because individual sensors pool responses originating from several centimeter of cortex, containing neural populations with widely varying spatial tuning. Here, we introduce a forward-modeling approach in which pRFs estimated from fMRI data are used to predict MEG sensor responses. Subjects viewed contrast-reversing bar stimuli sweeping across the visual field in separate fMRI and MEG sessions. Individual subject's pRFs were modeled on the cortical surface at the millimeter scale using the fMRI data. We then predicted cortical time series and projected these predictions to MEG sensors using a biophysical MEG forward model, accounting for the pooling across cortex. We compared the predicted MEG responses to observed visually evoked steady-state responses measured in the MEG session. We found that pRF parameters estimated by fMRI could explain a substantial fraction of the variance in steady-state MEG sensor responses (up to 60% in individual sensors). Control analyses in which we artificially perturbed either pRF size or pRF position reduced MEG prediction accuracy, indicating that MEG data are sensitive to pRF properties derived from fMRI. Our model provides a quantitative approach to link fMRI and MEG measurements, thereby enabling advances in our understanding of spatiotemporal dynamics in human visual field maps.

Imaging faster neural dynamics with fast fMRI: A need for updated models of the hemodynamic response

Fast fMRI enables the detection of neural dynamics over timescales of hundreds of milliseconds, suggesting it may provide a new avenue for studying subsecond neural processes in the human brain. The magnitudes of these fast fMRI dynamics are far greater than predicted by canonical models of the hemodynamic response. Several studies have established nonlinear properties of the hemodynamic response that have significant implications for fast fMRI. We first review nonlinear properties of the hemodynamic response function that may underlie fast fMRI signals. We then illustrate the breakdown of canonical hemodynamic response models in the context of fast neural dynamics. We will then argue that the canonical hemodynamic response function is not likely to reflect the BOLD response to neuronal activity driven by sparse or naturalistic stimuli or perhaps to spontaneous neuronal fluctuations in the resting state. These properties suggest that fast fMRI is capable of tracking surprisingly fast neuronal dynamics, and we discuss the neuroscientific questions that could be addressed using this approach.

Is Developmental Dyslexia Due to a Visual and Not a Phonological Impairment?

It is a widely held belief that developmental dyslexia (DD) is a phonological disorder in which readers have difficulty associating graphemes with their corresponding phonemes. In contrast, the magnocellular theory of dyslexia assumes that DD is a visual disorder caused by dysfunctional magnocellular neural pathways. The review explores arguments for and against these theories. Recent results have shown that DD is caused by (1) a reduced ability to simultaneously recognize sequences of letters that make up words, (2) longer fixation times required to simultaneously recognize strings of letters, and (3) amplitudes of saccades that do not match the number of simultaneously recognized letters. It was shown that pseudowords that could not be recognized simultaneously were recognized almost without errors when the fixation time was extended. However, there is an individual maximum number of letters that each reader with DD can recognize simultaneously. Findings on the neurobiological basis of temporal summation have shown that a necessary prolongation of fixation times is due to impaired processing mechanisms of the visual system, presumably involving magnocells and parvocells. An area in the mid-fusiform gyrus also appears to play a significant role in the ability to simultaneously recognize words and pseudowords. The results also contradict the assumption that DD is due to a lack of eye movement control. The present research does not support the assumption that DD is caused by a phonological disorder but shows that DD is due to a visual processing dysfunction.

Bayesian inference with incomplete knowledge explains perceptual confidence and its deviations from accuracy

In perceptual decisions, subjects infer hidden states of the environment based on noisy sensory information. Here we show that both choice and its associated confidence are explained by a Bayesian framework based on partially observable Markov decision processes (POMDPs). We test our model on monkeys performing a direction-discrimination task with post-decision wagering, demonstrating that the model explains objective accuracy and predicts subjective confidence. Further, we show that the model replicates well-known discrepancies of confidence and accuracy, including the hard-easy effect, opposing effects of stimulus variability on confidence and accuracy, dependence of confidence ratings on simultaneous or sequential reports of choice and confidence, apparent difference between choice and confidence sensitivity, and seemingly disproportionate influence of choice-congruent evidence on confidence. These effects may not be signatures of sub-optimal inference or discrepant computational processes for choice and confidence. Rather, they arise in Bayesian inference with incomplete knowledge of the environment.

Holistic face recognition is an emergent phenomenon of spatial processing in face-selective regions

Spatial processing by receptive fields is a core property of the visual system. However, it is unknown how spatial processing in high-level regions contributes to recognition behavior. As face inversion is thought to disrupt typical holistic processing of information in faces, we mapped population receptive fields (pRFs) with upright and inverted faces in the human visual system. Here we show that in face-selective regions, but not primary visual cortex, pRFs and overall visual field coverage are smaller and shifted downward in response to face inversion. From these measurements, we successfully predict the relative behavioral detriment of face inversion at different positions in the visual field. This correspondence between neural measurements and behavior demonstrates how spatial processing in face-selective regions may enable holistic perception. These results not only show that spatial processing in high-level visual regions is dynamically used towards recognition, but also suggest a powerful approach for bridging neural computations by receptive fields to behavior.

Remembering the Past to See the Future

In addition to the role that our visual system plays in determining what we are seeing right now, visual computations contribute in important ways to predicting what we will see next. While the role of memory in creating future predictions is often overlooked, efficient predictive computation requires the use of information about the past to estimate future events. In this article, we introduce a framework for understanding the relationship between memory and visual prediction and review the two classes of mechanisms that the visual system relies on to create future predictions. We also discuss the principles that define the mapping from predictive computations to predictive mechanisms and how downstream brain areas interpret the predictive signals computed by the visual system. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Pinpointing the neural signatures of single-exposure visual recognition memory

Memories of the images that we have seen are thought to be reflected in the reduction of neural responses in high-level visual areas such as inferotemporal (IT) cortex, a phenomenon known as repetition suppression (RS). We challenged this hypothesis with a task that required rhesus monkeys to report whether images were novel or repeated while ignoring variations in contrast, a stimulus attribute that is also known to modulate the overall IT response. The monkeys' behavior was largely contrast invariant, contrary to the predictions of an RS-inspired decoder, which could not distinguish responses to images that are repeated from those that are of lower contrast. However, the monkeys' behavioral patterns were well predicted by a linearly decodable variant in which the total spike count was corrected for contrast modulation. These results suggest that the IT neural activity pattern that best aligns with single-exposure visual recognition memory behavior is not RS but rather sensory referenced suppression: reductions in IT population response magnitude, corrected for sensory modulation.

Movies and narratives as naturalistic stimuli in neuroimaging

Using movies and narratives as naturalistic stimuli in human neuroimaging studies has yielded significant advances in understanding of cognitive and emotional functions. The relevant literature was reviewed, with emphasis on how the use of naturalistic stimuli has helped advance scientific understanding of human memory, attention, language, emotions, and social cognition in ways that would have been difficult otherwise. These advances include discovering a cortical hierarchy of temporal receptive windows, which supports processing of dynamic information that accumulates over several time scales, such as immediate reactions vs. slowly emerging patterns in social interactions. Naturalistic stimuli have also helped elucidate how the hippocampus supports segmentation and memorization of events in day-to-day life and have afforded insights into attentional brain mechanisms underlying our ability to adopt specific perspectives during natural viewing. Further, neuroimaging studies with naturalistic stimuli have revealed the role of the default-mode network in narrative-processing and in social cognition. Finally, by robustly eliciting genuine emotions, these stimuli have helped elucidate the brain basis of both basic and social emotions apparently manifested as highly overlapping yet distinguishable patterns of brain activity.

The role of action intentionality and effector in the subjective expansion of temporal duration after saccadic eye movements

Visual perception is based on periods of stable fixation separated by saccadic eye movements. Although naive perception seems stable (in space) and continuous (in time), laboratory studies have demonstrated that events presented around the time of saccades are misperceived spatially and temporally. Saccadic chronostasis, the "stopped clock illusion", represents one such temporal distortion in which the movement of the clock hand after the saccade is perceived as lasting longer than usual. Multiple explanations for chronostasis have been proposed including action-backdating, temporal binding of the action towards the moment of its effect ("intentional binding") and post-saccadic temporal dilation. The current study aimed to resolve this debate by using different types of action (keypress vs saccade) and varying the intentionality of the action. We measured both perceived onset of the motor action and perceived onset of an auditory tone presented at different delays after the keypress/saccade. The results showed intentional binding for the keypress action, with perceived motor onset shifted forwards in time and the time of the tone shifted backwards. Saccades resulted in the opposite pattern, showing temporal expansion rather than compression, especially with cued saccades. The temporal illusion was modulated by intentionality of the movement. Our findings suggest that saccadic chronostasis is not solely dependent on a backward shift in perceived saccade onset, but instead reflects a temporal dilation. This percept of an effectively "longer" period at the beginning of a new fixation may reflect the pattern of suppressed, and then enhanced, visual processing around the time of saccades.

A recurrent circuit implements normalization, simulating the dynamics of V1 activity

The normalization model has been applied to explain neural activity in diverse neural systems including primary visual cortex (V1). The model's defining characteristic is that the response of each neuron is divided by a factor that includes a weighted sum of activity of a pool of neurons. Despite the success of the normalization model, there are three unresolved issues. 1) Experimental evidence supports the hypothesis that normalization in V1 operates via recurrent amplification, i.e., amplifying weak inputs more than strong inputs. It is unknown how normalization arises from recurrent amplification. 2) Experiments have demonstrated that normalization is weighted such that each weight specifies how one neuron contributes to another's normalization pool. It is unknown how weighted normalization arises from a recurrent circuit. 3) Neural activity in V1 exhibits complex dynamics, including gamma oscillations, linked to normalization. It is unknown how these dynamics emerge from normalization. Here, a family of recurrent circuit models is reported, each of which comprises coupled neural integrators to implement normalization via recurrent amplification with arbitrary normalization weights, some of which can recapitulate key experimental observations of the dynamics of neural activity in V1.

Auditory and tactile frequency representations are co-embedded in modality-defined cortical sensory systems

Sensory information is represented and elaborated in hierarchical cortical systems that are thought to be dedicated to individual sensory modalities. This traditional view of sensory cortex organization has been challenged by recent evidence of multimodal responses in primary and association sensory areas. Although it is indisputable that sensory areas respond to multiple modalities, it remains unclear whether these multimodal responses reflect selective information processing for particular stimulus features. Here, we used fMRI adaptation to identify brain regions that are sensitive to the temporal frequency information contained in auditory, tactile, and audiotactile stimulus sequences. A number of brain regions distributed over the parietal and temporal lobes exhibited frequency-selective temporal response modulation for both auditory and tactile stimulus events, as indexed by repetition suppression effects. A smaller set of regions responded to crossmodal adaptation sequences in a frequency-dependent manner. Despite an extensive overlap of multimodal frequency-selective responses across the parietal and temporal lobes, representational similarity analysis revealed a cortical "regional landscape" that clearly reflected distinct somatosensory and auditory processing systems that converged on modality-invariant areas. These structured relationships between brain regions were also evident in spontaneous signal fluctuation patterns measured at rest. Our results reveal that multimodal processing in human cortex can be feature-specific and that multimodal frequency representations are embedded in the intrinsically hierarchical organization of cortical sensory systems.

Diverse Temporal Dynamics of Repetition Suppression Revealed by Intracranial Recordings in the Human Ventral Temporal Cortex

Repeated stimulus presentations commonly produce decreased neural responses-a phenomenon known as repetition suppression (RS) or adaptation-in ventral temporal cortex (VTC) of humans and nonhuman primates. However, the temporal features of RS in human VTC are not well understood. To fill this gap in knowledge, we utilized the precise spatial localization and high temporal resolution of electrocorticography (ECoG) from nine human subjects implanted with intracranial electrodes in the VTC. The subjects viewed nonrepeated and repeated images of faces with long-lagged intervals and many intervening stimuli between repeats. We report three main findings: 1) robust RS occurs in VTC for activity in high-frequency broadband (HFB), but not lower-frequency bands; 2) RS of the HFB signal is associated with lower peak magnitude (PM), lower total responses, and earlier peak responses; and 3) RS effects occur early within initial stages of stimulus processing and persist for the entire stimulus duration. We discuss these findings in the context of early and late components of visual perception, as well as theoretical models of repetition suppression.

A validation framework for neuroimaging software: The case of population receptive fields

Neuroimaging software methods are complex, making it a near certainty that some implementations will contain errors. Modern computational techniques (i.e., public code and data repositories, continuous integration, containerization) enable the reproducibility of the analyses and reduce coding errors, but they do not guarantee the scientific validity of the results. It is difficult, nay impossible, for researchers to check the accuracy of software by reading the source code; ground truth test datasets are needed. Computational reproducibility means providing software so that for the same input anyone obtains the same result, right or wrong. Computational validity means obtaining the right result for the ground-truth test data. We describe a framework for validating and sharing software implementations, and we illustrate its usage with an example application: population receptive field (pRF) methods for functional MRI data. The framework is composed of three main components implemented with containerization methods to guarantee computational reproducibility. In our example pRF application, those components are: (1) synthesis of fMRI time series from ground-truth pRF parameters, (2) implementation of four public pRF analysis tools and standardization of inputs and outputs, and (3) report creation to compare the results with the ground truth parameters. The framework was useful in identifying realistic conditions that lead to imperfect parameter recovery in all four pRF implementations, that would remain undetected using classic validation methods. We provide means to mitigate these problems in future experiments. A computational validation framework supports scientific rigor and creativity, as opposed to the oft-repeated suggestion that investigators rely upon a few agreed upon packages. We hope that the framework will be helpful to validate other critical neuroimaging algorithms, as having a validation framework helps (1) developers to build new software, (2) research scientists to verify the software's accuracy, and (3) reviewers to evaluate the methods used in publications and grants.

Neural representations of perceptual color experience in the human ventral visual pathway

Color is a perceptual construct that arises from neural processing in hierarchically organized cortical visual areas. Previous research, however, often failed to distinguish between neural responses driven by stimulus chromaticity versus perceptual color experience. An unsolved question is whether the neural responses at each stage of cortical processing represent a physical stimulus or a color we see. The present study dissociated the perceptual domain of color experience from the physical domain of chromatic stimulation at each stage of cortical processing by using a switch rivalry paradigm that caused the color percept to vary over time without changing the retinal stimulation. Using functional MRI (fMRI) and a model-based encoding approach, we found that neural representations in higher visual areas, such as V4 and VO1, corresponded to the perceived color, whereas responses in early visual areas V1 and V2 were modulated by the chromatic light stimulus rather than color perception. Our findings support a transition in the ascending human ventral visual pathway, from a representation of the chromatic stimulus at the retina in early visual areas to responses that correspond to perceptually experienced colors in higher visual areas.

Constructing and Forgetting Temporal Context in the Human Cerebral Cortex

How does information from seconds earlier affect neocortical responses to new input? We found that when two groups of participants heard the same sentence in a narrative, preceded by different contexts, the neural responses of each group were initially different but gradually fell into alignment. We observed a hierarchical gradient: sensory cortices aligned most quickly, followed by mid-level regions, while some higher-order cortical regions took more than 10 seconds to align. What computations explain this hierarchical temporal organization? Linear integration models predict that regions that are slower to integrate new information should also be slower to forget old information. However, we found that higher-order regions could rapidly forget prior context. The data from the cortical hierarchy were instead captured by a model in which each region maintains a temporal context representation that is nonlinearly integrated with input at each moment, and this integration is gated by local prediction error.

Center-surround velocity-based segmentation: Speed, eccentricity, and timing of visual stimuli interact to determine interocular dominance

We used a novel method to capture the spatial dominance pattern of competing motion fields at rivalry onset. When rivaling velocities were different, the participants reported center-surround segmentation: The slower stimuli often dominated in the center while faster motion persisted along the borders. The size of the central static/slow field scaled with the stimulus size. The central dominance was time-locked to the static stimulus onset but was disrupted if the dynamic stimulus was presented later. We then used the same stimuli as masks in an interocular suppression paradigm. The local suppression strengths were probed with targets at different eccentricities. Consistent with the center-surround segmentation, target speed and location interacted with mask velocities. Specifically, suppression power of the slower masks was nonhomogenous with eccentricity, providing a potential explanation for center-surround velocity-based segmentation. This interaction of speed, eccentricity, and timing has implications for motion processing and interocular suppression. The influence of different masks on which target features get suppressed predicts that some "unconscious effects" are not generalizable across masks and, thus, need to be replicated under various masking conditions.

A Network of Topographic Maps in Human Association Cortex Hierarchically Transforms Visual Timing-Selective Responses

Accurately timing sub-second sensory events is crucial when perceiving our dynamic world. This ability allows complex human behaviors that require timing-dependent multisensory integration and action planning. Such behaviors include perception and performance of speech, music, driving, and many sports. How are responses to sensory event timing processed for multisensory integration and action planning? We measured responses to viewing systematically changing visual event timing using ultra-high-field fMRI. We analyzed these responses with neural population response models selective for event duration and frequency, following behavioral, computational, and macaque action planning results and comparisons to alternative models. We found systematic local changes in timing preferences (recently described in supplementary motor area) in an extensive network of topographic timing maps, mirroring sensory cortices and other quantity processing networks. These timing maps were partially left lateralized and widely spread, from occipital visual areas through parietal multisensory areas to frontal action planning areas. Responses to event duration and frequency were closely linked. As in sensory cortical maps, response precision varied systematically with timing preferences, and timing selectivity systematically varied between maps. Progressing from posterior to anterior maps, responses to multiple events were increasingly integrated, response selectivity narrowed, and responses focused increasingly on the middle of the presented timing range. These timing maps largely overlap with numerosity and visual field map networks. In both visual timing map and visual field map networks, selective responses and topographic map organization may facilitate hierarchical transformations by allowing neural populations to interact over minimal distances.

•

Many brain areas show neural responses to specific ranges of visual event timing

•

Timing preferences change gradually in these areas, forming topographic timing maps

•

Neural response properties hierarchically transform from visual to premotor areas

•

Timing, numerosity, and visual field map networks are distinct but largely overlap

Population spatial frequency tuning in human early visual cortex

Neurons within early visual cortex are selective for basic image statistics, including spatial frequency. However, these neurons are thought to act as band-pass filters, with the window of spatial frequency sensitivity varying across the visual field and across visual areas. Although a handful of previous functional (f)MRI studies have examined human spatial frequency sensitivity using conventional designs and analysis methods, these measurements are time consuming and fail to capture the precision of spatial frequency tuning (bandwidth). In this study, we introduce a model-driven approach to fMRI analyses that allows for fast and efficient estimation of population spatial frequency tuning (pSFT) for individual voxels. Blood oxygen level-dependent (BOLD) responses within early visual cortex were acquired while subjects viewed a series of full-field stimuli that swept through a large range of spatial frequency content. Each stimulus was generated by band-pass filtering white noise with a central frequency that changed periodically between a minimum of 0.5 cycles/degree (cpd) and a maximum of 12 cpd. To estimate the underlying frequency tuning of each voxel, we assumed a log-Gaussian pSFT and optimized the parameters of this function by comparing our model output against the measured BOLD time series. Consistent with previous studies, our results show that an increase in eccentricity within each visual area is accompanied by a drop in the peak spatial frequency of the pSFT. Moreover, we found that pSFT bandwidth depends on eccentricity and is correlated with the pSFT peak; populations with lower peaks possess broader bandwidths in logarithmic scale, whereas in linear scale this relationship is reversed.NEW & NOTEWORTHY Spatial frequency selectivity is a hallmark property of early visuocortical neurons, and mapping these sensitivities gives us crucial insight into the hierarchical organization of information within visual areas. Due to technical obstacles, we lack a comprehensive picture of the properties of this sensitivity in humans. Here, we introduce a new method, coined population spatial frequency tuning mapping, which circumvents the limitations of the conventional neuroimaging methods, yielding a fuller visuocortical map of spatial frequency sensitivity.

Temporal tuning of repetition suppression across the visual cortex

The visual system adapts to its recent history. A phenomenon related to this is repetition suppression (RS), a reduction in neural responses to repeated compared with nonrepeated visual input. An intriguing hypothesis is that the timescale over which RS occurs across the visual hierarchy is tuned to the temporal statistics of visual input features, which change rapidly in low-level areas but are more stable in higher level areas. Here, we tested this hypothesis by studying the influence of the temporal lag between successive visual stimuli on RS throughout the visual system using functional (f)MRI. Twelve human volunteers engaged in four fMRI sessions in which we characterized the blood oxygen level-dependent response to pairs of repeated and nonrepeated natural images with interstimulus intervals (ISI) ranging from 50 to 1,000 ms to quantify the temporal tuning of RS along the posterior-anterior axis of the visual system. As expected, RS was maximal for short ISIs and decayed with increasing ISI. Crucially, however, and against our hypothesis, RS decayed at a similar rate in early and late visual areas. This finding challenges the prevailing view that the timescale of RS increases along the posterior-anterior axis of the visual system and suggests that RS is not tuned to temporal input regularities.NEW & NOTEWORTHY Visual areas show reduced neural responses to repeated compared with nonrepeated visual input, a phenomenon termed repetition suppression (RS). Here we show that RS decays at a similar rate in low- and high-level visual areas, suggesting that the short-term decay of RS across the visual hierarchy is not tuned to temporal input regularities. This may limit the specificity with which the mechanisms underlying RS could optimize the processing of input features across the visual hierarchy.

Predicting neuronal dynamics with a delayed gain control model

Visual neurons respond to static images with specific dynamics: neuronal responses sum sub-additively over time, reduce in amplitude with repeated or sustained stimuli (neuronal adaptation), and are slower at low stimulus contrast. Here, we propose a simple model that predicts these seemingly disparate response patterns observed in a diverse set of measurements-intracranial electrodes in patients, fMRI, and macaque single unit spiking. The model takes a time-varying contrast time course of a stimulus as input, and produces predicted neuronal dynamics as output. Model computation consists of linear filtering, expansive exponentiation, and a divisive gain control. The gain control signal relates to but is slower than the linear signal, and this delay is critical in giving rise to predictions matched to the observed dynamics. Our model is simpler than previously proposed related models, and fitting the model to intracranial EEG data uncovers two regularities across human visual field maps: estimated linear filters (temporal receptive fields) systematically differ across and within visual field maps, and later areas exhibit more rapid and substantial gain control. The model is further generalizable to account for dynamics of contrast-dependent spike rates in macaque V1, and amplitudes of fMRI BOLD in human V1.

Visual novelty, curiosity, and intrinsic reward in machine learning and the brain

A strong preference for novelty emerges in infancy and is prevalent across the animal kingdom. When incorporated into reinforcement-based machine learning algorithms, visual novelty can act as an intrinsic reward signal that vastly increases the efficiency of exploration and expedites learning, particularly in situations where external rewards are difficult to obtain. Here we review parallels between recent developments in novelty-driven machine learning algorithms and our understanding of how visual novelty is computed and signaled in the primate brain. We propose that in the visual system, novelty representations are not configured with the principal goal of detecting novel objects, but rather with the broader goal of flexibly generalizing novelty information across different states in the service of driving novelty-based learning.