Successor-like representation guides the prediction of future events in human visual cortex and hippocampus

Predictable and unpredictable deviance detection in the human hippocampus and amygdala

Our brains extract structure from the environment and form predictions given past experience. Predictive circuits have been identified in wide-spread cortical regions. However, the contribution of medial temporal structures in predictions remains under-explored. The hippocampus underlies sequence detection and is sensitive to novel stimuli, sufficient to gain access to memory, while the amygdala to novelty. Yet, their electrophysiological profiles in detecting predictable and unpredictable deviant auditory events remain unknown. Here, we hypothesized that the hippocampus would be sensitive to predictability, while the amygdala to unexpected deviance. We presented epileptic patients undergoing presurgical monitoring with standard and deviant sounds, in predictable or unpredictable contexts. Onsets of auditory responses and unpredictable deviance effects were detected earlier in the temporal cortex compared with the amygdala and hippocampus. Deviance effects in 1-20 Hz local field potentials were detected in the lateral temporal cortex, irrespective of predictability. The amygdala showed stronger deviance in the unpredictable context. Low-frequency deviance responses in the hippocampus (1-8 Hz) were observed in the predictable but not in the unpredictable context. Our results reveal a distributed network underlying the generation of auditory predictions and suggest that the neural basis of sensory predictions and prediction error signals needs to be extended.

The Hippocampus in Pigeons Contributes to the Model-Based Valuation and the Relationship between Temporal Context States

Model-based decision-making guides organism behavior by the representation of the relationships between different states. Previous studies have shown that the mammalian hippocampus (Hp) plays a key role in learning the structure of relationships among experiences. However, the hippocampal neural mechanisms of birds for model-based learning have rarely been reported. Here, we trained six pigeons to perform a two-step task and explore whether their Hp contributes to model-based learning. Behavioral performance and hippocampal multi-channel local field potentials (LFPs) were recorded during the task. We estimated the subjective values using a reinforcement learning model dynamically fitted to the pigeon's choice of behavior. The results show that the model-based learner can capture the behavioral choices of pigeons well throughout the learning process. Neural analysis indicated that high-frequency (12-100 Hz) power in Hp represented the temporal context states. Moreover, dynamic correlation and decoding results provided further support for the high-frequency dependence of model-based valuations. In addition, we observed a significant increase in hippocampal neural similarity at the low-frequency band (1-12 Hz) for common temporal context states after learning. Overall, our findings suggest that pigeons use model-based inferences to learn multi-step tasks, and multiple LFP frequency bands collaboratively contribute to model-based learning. Specifically, the high-frequency (12-100 Hz) oscillations represent model-based valuations, while the low-frequency (1-12 Hz) neural similarity is influenced by the relationship between temporal context states. These results contribute to our understanding of the neural mechanisms underlying model-based learning and broaden the scope of hippocampal contributions to avian behavior.

Traveling waves shape neural population dynamics enabling predictions and internal model updating

The brain generates predictions based on statistical regularities in our environment. However, it is unclear how predictions are optimized through iterative interactions with the environment. Because traveling waves (TWs) propagate across the cortex shaping neural excitability, they can carry information to serve predictive processing. Using human intracranial recordings, we show that anterior-to-posterior alpha TWs correlated with prediction strength. Learning about priors altered neural state space trajectories, and how much it altered correlated with trial-by-trial prediction strength. Learning involved mismatches between predictions and sensory evidence triggering alpha-phase resets in lateral temporal cortex, accompanied by stronger alpha phase-high gamma amplitude coupling and high-gamma power. The mismatch initiated posterior-to-anterior alpha TWs and change in the subsequent trial's state space trajectory, facilitating model updating. Our findings suggest a vital role of alpha TWs carrying both predictions to sensory cortex and mismatch signals to frontal cortex for trial-by-trial fine-tuning of predictive models.

Neural patterns of conscious visual awareness in the Riddoch syndrome

The Riddoch syndrome is one in which patients blinded by lesions to their primary visual cortex can consciously perceive visual motion in their blind field, an ability that correlates with activity in motion area V5. Our assessment of the characteristics of this syndrome in patient ST, using multimodal MRI, showed that: 1. ST's V5 is intact, receives direct subcortical input, and decodable neural patterns emerge in it only during the conscious perception of visual motion; 2. moving stimuli activate medial visual areas but, unless associated with decodable V5 activity, they remain unperceived; 3. ST's high confidence ratings when discriminating motion at chance levels, is associated with inferior frontal gyrus activity. Finally, we report that ST's Riddoch Syndrome results in hallucinatory motion with hippocampal activity as a correlate. Our results shed new light on perceptual experiences associated with this syndrome and on the neural determinants of conscious visual experience.

Gamma oscillations and episodic memory

Enhanced gamma oscillatory activity (30-80 Hz) accompanies the successful formation and retrieval of episodic memories. While this co-occurrence is well documented, the mechanistic contributions of gamma oscillatory activity to episodic memory remain unclear. Here, we review how gamma oscillatory activity may facilitate spike timing-dependent plasticity, neural communication, and sequence encoding/retrieval, thereby ensuring the successful formation and/or retrieval of an episodic memory. Based on the evidence reviewed, we propose that multiple, distinct forms of gamma oscillation can be found within the canonical gamma band, each of which has a complementary role in the neural processes listed above. Further exploration of these theories using causal manipulations may be key to elucidating the relevance of gamma oscillatory activity to episodic memory.

Prior knowledge about events depicted in scenes decreases oculomotor exploration

The visual input that the eyes receive usually contains temporally continuous information about unfolding events. Therefore, humans can accumulate knowledge about their current environment. Typical studies on scene perception, however, involve presenting multiple unrelated images and thereby render this accumulation unnecessary. Our study, instead, facilitated it and explored its effects. Specifically, we investigated how recently-accumulated prior knowledge affects gaze behavior. Participants viewed sequences of static film frames that contained several 'context frames' followed by a 'critical frame'. The context frames showed either events from which the situation depicted in the critical frame naturally followed, or events unrelated to this situation. Therefore, participants viewed identical critical frames while possessing prior knowledge that was either relevant or irrelevant to the frames' content. In the former case, participants' gaze behavior was slightly more exploratory, as revealed by seven gaze characteristics we analyzed. This result demonstrates that recently-gained prior knowledge reduces exploratory eye movements.

Visual Information Is Predictively Encoded in Occipital Alpha/Low-Beta Oscillations

Hierarchical predictive coding networks are a general model of sensory processing in the brain. Under neural delays, these networks have been suggested to naturally generate oscillatory activity in approximately the α frequency range (∼8-12 Hz). This suggests that α oscillations, a prominent feature of EEG recordings, may be a spectral "fingerprint" of predictive sensory processing. Here, we probed this possibility by investigating whether oscillations over the visual cortex predictively encode visual information. Specifically, we examined whether their power carries information about the position of a moving stimulus, in a temporally predictive fashion. In two experiments (N = 32, 18 female; N = 34, 17 female), participants viewed an apparent-motion stimulus moving along a circular path while EEG was recorded. To investigate the encoding of stimulus-position information, we developed a method of deriving probabilistic spatial maps from oscillatory power estimates. With this method, we demonstrate that it is possible to reconstruct the trajectory of a moving stimulus from α/low-β oscillations, tracking its position even across unexpected motion reversals. We also show that future position representations are activated in the absence of direct visual input, demonstrating that temporally predictive mechanisms manifest in α/β band oscillations. In a second experiment, we replicate these findings and show that the encoding of information in this range is not driven by visual entrainment. By demonstrating that occipital α/β oscillations carry stimulus-related information, in a temporally predictive fashion, we provide empirical evidence of these rhythms as a spectral "fingerprint" of hierarchical predictive processing in the human visual system.SIGNIFICANCE STATEMENT "Hierarchical predictive coding" is a general model of sensory information processing in the brain. When in silico predictive coding models are constrained by neural transmission delays, their activity naturally oscillates in roughly the α range (∼8-12 Hz). Using time-resolved EEG decoding, we show that neural rhythms in this approximate range (α/low-β) over the human visual cortex predictively encode the position of a moving stimulus. From the amplitude of these oscillations, we are able to reconstruct the stimulus' trajectory, revealing signatures of temporally predictive processing. This provides direct neural evidence linking occipital α/β rhythms to predictive visual processing, supporting the emerging view of such oscillations as a potential spectral "fingerprint" of hierarchical predictive processing in the human visual system.

The brain hierarchically represents the past and future during multistep anticipation

Memory for temporal structure enables both planning of future events and retrospection of past events. We investigated how the brain flexibly represents extended temporal sequences into the past and future during anticipation. Participants learned sequences of environments in immersive virtual reality. Pairs of sequences had the same environments in a different order, enabling context-specific learning. During fMRI, participants anticipated upcoming environments multiple steps into the future in a given sequence. Temporal structure was represented in the hippocampus and across visual regions (1) bidirectionally, with graded representations into the past and future and (2) hierarchically, with further events into the past and future represented in successively more anterior brain regions. Further, context-specific predictions were prioritized in the forward but not backward direction. Together, this work sheds light on how we flexibly represent sequential structure to enable planning over multiple timescales.

Predictive neural representations of naturalistic dynamic input

Adaptive behavior such as social interaction requires our brain to predict unfolding external dynamics. While theories assume such dynamic prediction, empirical evidence is limited to static snapshots and indirect consequences of predictions. We present a dynamic extension to representational similarity analysis that uses temporally variable models to capture neural representations of unfolding events. We applied this approach to source-reconstructed magnetoencephalography (MEG) data of healthy human subjects and demonstrate both lagged and predictive neural representations of observed actions. Predictive representations exhibit a hierarchical pattern, such that high-level abstract stimulus features are predicted earlier in time, while low-level visual features are predicted closer in time to the actual sensory input. By quantifying the temporal forecast window of the brain, this approach allows investigating predictive processing of our dynamic world. It can be applied to other naturalistic stimuli (e.g., film, soundscapes, music, motor planning/execution, social interaction) and any biosignal with high temporal resolution.

Optogenetic manipulation of inhibitory interneurons can be used to validate a model of spatiotemporal sequence learning

The brain uses temporal information to link discrete events into memory structures supporting recognition, prediction, and a wide variety of complex behaviors. It is still an open question how experience-dependent synaptic plasticity creates memories including temporal and ordinal information. Various models have been proposed to explain how this could work, but these are often difficult to validate in a living brain. A recent model developed to explain sequence learning in the visual cortex encodes intervals in recurrent excitatory synapses and uses a learned offset between excitation and inhibition to generate precisely timed "messenger" cells that signal the end of an instance of time. This mechanism suggests that the recall of stored temporal intervals should be particularly sensitive to the activity of inhibitory interneurons that can be easily targeted in vivo with standard optogenetic tools. In this work we examined how simulated optogenetic manipulations of inhibitory cells modifies temporal learning and recall based on these mechanisms. We show that disinhibition and excess inhibition during learning or testing cause characteristic errors in recalled timing that could be used to validate the model in vivo using either physiological or behavioral measurements.