Alpha-frequency feedback to early visual cortex orchestrates coherent naturalistic vision

Layer-specific spatiotemporal dynamics of feedforward and feedback in human visual object perception

Visual object perception is mediated by information flow between regions of the ventral visual stream along feedforward and feedback anatomical connections. However, feedforward and feedback signals during naturalistic vision are rapid and overlapping, complicating their identification and precise functional specification. Here we recorded human layer-specific fMRI responses to naturalistic object images in early visual cortex (EVC) and lateral occipital complex (LOC) to isolate feedforward and feedback information signals spatially by their cortical layer specific termination pattern. We combined these layer-specific fMRI responses with electroencephalography (EEG) responses for the same images to segregate feedforward and feedback signals in both time and space. Feedforward signals emerge early in the middle layers of EVC and LOC, followed by feedback signals in the superficial layer of both regions, and the deep layer of EVC. Comparing the identified dynamics in LOC to a visual deep neural network (DNN), revealed that early feedforward signals in LOC encode medium complexity features, whereas later feedback signals increase the representational format to high complexity features. Together this specifies the spatiotemporal dynamics and functional role of feedforward and feedback information flow mediating visual object perception.

Cortical alpha rhythms interpolate occluded motion from natural scene context

Tracking objects as they dynamically move in and out of sight is critical for parsing our everchanging real-world surroundings. Here, we explored how the interpolation of occluded object motion in natural scenes is mediated by top-down information flows expressed in cortical alpha rhythms. We recorded EEG while participants viewed videos of a person walking across a scene. We then used multivariate decoding on alpha-band responses to decode the direction of movement across the scene. In trials where the person was temporarily occluded, alpha dynamics interpolated the person's predicted movement. Critically, they did so in a context-dependent manner: When the scene context required the person to stop in front of an obstacle, alpha dynamics tracked the termination of motion during occlusion. As these effects were obtained with an orthogonal task at fixation, we conclude that alpha rhythms automatically interpolate occluded motion based on the contextual cues from the surrounding environment.NEW & NOTEWORTHY Inferring how objects continue to move during occlusion requires contextual cues from the surrounding environment. Such contextual information is incorporated via neural feedback linked to cortical alpha oscillations. Here, we demonstrate that alpha dynamics track the predicted movement of a person during occlusion, depending on scene context: Alpha oscillations not only track how the person moves when their path is unobstructed but also when they need to stop because of obstacles blocking their way.

Representational shifts from feedforward to feedback rhythms index phenomenological integration in naturalistic vision

How does the brain integrate complex and dynamic visual inputs into phenomenologically seamless percepts? Previous results demonstrate that when visual inputs are organized coherently across space and time, they are more strongly encoded in feedback-related alpha rhythms, and less strongly in feedforward-related gamma rhythms. Here, we tested whether this representational shift from feedforward to feedback rhythms is linked to the phenomenological experience of coherence. In an Electroencephalography (EEG) study, we manipulated the degree of spatiotemporal coherence by presenting two segments from the same video across visual hemifields, either synchronously or asynchronously (with a delay between segments). We asked participants whether they perceived the stimulus as coherent or incoherent. When stimuli were presented at the perceptual threshold (i.e., when the same stimulus was judged as coherent 50% of times), perception co-varied with stimulus coding across alpha and gamma rhythms: When stimuli were perceived as coherent, they were represented in alpha activity; when stimuli were perceived as incoherent, they were represented in gamma activity. Whether the same visual input is perceived as coherent or incoherent thus depends on representational shifts between feedback-related alpha and feedforward-related gamma rhythms.

Decoding the rhythmic representation and communication of visual contents

Rhythmic neural activity is considered essential for adaptively modulating responses in the visual system. In this opinion article we posit that visual brain rhythms also serve a key function in the representation and communication of visual contents. Collating a set of recent studies that used multivariate decoding methods on rhythmic brain signals, we highlight such rhythmic content representations in visual perception, imagery, and prediction. We argue that characterizing representations across frequency bands allows researchers to elegantly disentangle content transfer in feedforward and feedback directions. We further propose that alpha dynamics are central to content-specific feedback propagation in the visual system. We conclude that considering rhythmic content codes is pivotal for understanding information coding in vision and beyond.

Enhanced and idiosyncratic neural representations of personally typical scenes

Previous research shows that the typicality of visual scenes (i.e. if they are good examples of a category) determines how easily they can be perceived and represented in the brain. However, the unique visual diets individuals are exposed to across their lifetimes should sculpt very personal notions of typicality. Here, we thus investigated whether scenes that are more typical to individual observers are more accurately perceived and represented in the brain. We used drawings to enable participants to describe typical scenes (e.g. a kitchen) and converted these drawings into three-dimensional renders. These renders were used as stimuli in a scene categorization task, during which we recorded electroencephalography (EEG). In line with previous findings, categorization was most accurate for renders resembling the typical scene drawings of individual participants. Our EEG analyses reveal two critical insights on how these individual differences emerge on the neural level. First, personally typical scenes yielded enhanced neural representations from around 200 ms after onset. Second, personally typical scenes were represented in idiosyncratic ways, with reduced dependence on high-level visual features. We interpret these findings in a predictive processing framework, where individual differences in internal models of scene categories formed through experience shape visual analysis in idiosyncratic ways.

Comparison of whole-brain task-modulated functional connectivity methods for fMRI task connectomics

Higher brain functions require flexible integration of information across widely distributed brain regions depending on the task context. Resting-state functional magnetic resonance imaging (fMRI) has provided substantial insight into large-scale intrinsic brain network organisation, yet the principles of rapid context-dependent reconfiguration of that intrinsic network organisation are much less understood. A major challenge for task connectome mapping is the absence of a gold standard for deriving whole-brain task-modulated functional connectivity matrices. Here, we perform biophysically realistic simulations to control the ground-truth task-modulated functional connectivity over a wide range of experimental settings. We reveal the best-performing methods for different types of task designs and their fundamental limitations. Importantly, we demonstrate that rapid (100 ms) modulations of oscillatory neuronal synchronisation can be recovered from sluggish haemodynamic fluctuations even at typically low fMRI temporal resolution (2 s). Finally, we provide practical recommendations on task design and statistical analysis to foster task connectome mapping.

Comparison of whole-brain task-modulated functional connectivity methods for fMRI task connectomics

Higher brain functions require flexible integration of information across widely distributed brain regions depending on the task context. Resting-state functional magnetic resonance imaging (fMRI) has provided substantial insight into large-scale intrinsic brain network organisation, yet the principles of rapid context-dependent reconfiguration of that intrinsic network organisation are much less understood. A major challenge for task connectome mapping is the absence of a gold standard for deriving whole-brain task-modulated functional connectivity matrices. Here, we perform biophysically realistic simulations to control the ground-truth task-modulated functional connectivity over a wide range of experimental settings. We reveal the best-performing methods for different types of task designs and their fundamental limitations. Importantly, we demonstrate that rapid (100 ms) modulations of oscillatory neuronal synchronisation can be recovered from sluggish haemodynamic fluctuations even at typically low fMRI temporal resolution (2 s). Finally, we provide practical recommendations on task design and statistical analysis to foster task connectome mapping.

Representations of imaginary scenes and their properties in cortical alpha activity

Imagining natural scenes enables us to engage with a myriad of simulated environments. How do our brains generate such complex mental images? Recent research suggests that cortical alpha activity carries information about individual objects during visual imagery. However, it remains unclear if more complex imagined contents such as natural scenes are similarly represented in alpha activity. Here, we answer this question by decoding the contents of imagined scenes from rhythmic cortical activity patterns. In an EEG experiment, participants imagined natural scenes based on detailed written descriptions, which conveyed four complementary scene properties: openness, naturalness, clutter level and brightness. By conducting classification analyses on EEG power patterns across neural frequencies, we were able to decode both individual imagined scenes as well as their properties from the alpha band, showing that also the contents of complex visual images are represented in alpha rhythms. A cross-classification analysis between alpha power patterns during the imagery task and during a perception task, in which participants were presented images of the described scenes, showed that scene representations in the alpha band are partly shared between imagery and late stages of perception. This suggests that alpha activity mediates the top-down re-activation of scene-related visual contents during imagery.

Coherent categorical information triggers integration-related alpha dynamics

To create coherent visual experiences, the brain spatially integrates the complex and dynamic information it receives from the environment. We previously demonstrated that feedback-related alpha activity carries stimulus-specific information when two spatially and temporally coherent naturalistic inputs can be integrated into a unified percept. In this study, we sought to determine whether such integration-related alpha dynamics are triggered by categorical coherence in visual inputs. In an EEG experiment, we manipulated the degree of coherence by presenting pairs of videos from the same or different categories through two apertures in the left and right visual hemifields. Critically, video pairs could be video-level coherent (i.e., stem from the same video), coherent in their basic-level category, coherent in their superordinate category, or incoherent (i.e., stem from videos from two entirely different categories). We conducted multivariate classification analyses on rhythmic EEG responses to decode between the video stimuli in each condition. As the key result, we significantly decoded the video-level coherent and basic-level coherent stimuli, but not the superordinate coherent and incoherent stimuli, from cortical alpha rhythms. This suggests that alpha dynamics play a critical role in integrating information across space, and that cortical integration processes are flexible enough to accommodate information from different exemplars of the same basic-level category.NEW & NOTEWORTHY Our brain integrates dynamic inputs across the visual field to create coherent visual experiences. Such integration processes have previously been linked to cortical alpha dynamics. In this study, the integration-related alpha activity was observed not only when snippets from the same video were presented, but also when different video snippets from the same basic-level category were presented, highlighting the flexibility of neural integration processes.

Integrative processing in artificial and biological vision predicts the perceived beauty of natural images

Previous research shows that the beauty of natural images is already determined during perceptual analysis. However, it is unclear which perceptual computations give rise to the perception of beauty. Here, we tested whether perceived beauty is predicted by spatial integration across an image, a perceptual computation that reduces processing demands by aggregating image parts into more efficient representations of the whole. We quantified integrative processing in an artificial deep neural network model, where the degree of integration was determined by the amount of deviation between activations for the whole image and its constituent parts. This quantification of integration predicted beauty ratings for natural images across four studies with different stimuli and designs. In a complementary functional magnetic resonance imaging study, we show that integrative processing in human visual cortex similarly predicts perceived beauty. Together, our results establish integration as a computational principle that facilitates perceptual analysis and thereby mediates the perception of beauty.

Distinct neural signatures underlying information maintenance and manipulation in working memory

Previous working memory research has demonstrated robust stimulus representations during memory maintenance in both voltage and alpha-band activity in electroencephalography. However, the exact functions of these 2 neural signatures have remained controversial. Here we systematically investigated their respective contributions to memory manipulation. Human participants either maintained a previously seen spatial location, or manipulated the location following a mental rotation cue over a delay. Using multivariate decoding, we observed robust location representations in low-frequency voltage and alpha-band oscillatory activity with distinct spatiotemporal dynamics: location representations were most evident in posterior channels in alpha-band activity, but were most prominent in the more anterior, central channels in voltage signals. Moreover, the temporal emergence of manipulated representation in central voltage preceded that in posterior alpha-band activity, suggesting that voltage might carry stimulus-specific source signals originated internally from anterior cortex, whereas alpha-band activity might reflect feedback signals in posterior cortex received from higher-order cortex. Lastly, while location representations in both signals were coded in a low-dimensional neural subspace, location representation in central voltage was higher-dimensional and underwent a representational transformation that exclusively predicted memory behavior. Together, these results highlight the crucial role of central voltage in working memory, and support functional distinctions between voltage and alpha-band activity.