Spatiotemporal Dynamics of Cortical Representations during and after Stimulus Presentation

The Time Course of Language Production as Revealed by Pattern Classification of MEG Sensor Data

Language production involves a complex set of computations, from conceptualization to articulation, which are thought to engage cascading neural events in the language network. However, recent neuromagnetic evidence suggests simultaneous meaning-to-speech mapping in picture naming tasks, as indexed by early parallel activation of frontotemporal regions to lexical semantic, phonological, and articulatory information. Here we investigate the time course of word production, asking to what extent such "earliness" is a distinctive property of the associated spatiotemporal dynamics. Using MEG, we recorded the neural signals of 34 human subjects (26 males) overtly naming 134 images from four semantic object categories (animals, foods, tools, clothes). Within each category, we covaried word length, as quantified by the number of syllables contained in a word, and phonological neighborhood density to target lexical and post-lexical phonological/phonetic processes. Multivariate pattern analyses searchlights in sensor space distinguished the stimulus-locked spatiotemporal responses to object categories early on, from 150 to 250 ms after picture onset, whereas word length was decoded in left frontotemporal sensors at 250-350 ms, followed by the latency of phonological neighborhood density (350-450 ms). Our results suggest a progression of neural activity from posterior to anterior language regions for the semantic and phonological/phonetic computations preparing overt speech, thus supporting serial cascading models of word production.SIGNIFICANCE STATEMENT Current psycholinguistic models make divergent predictions on how a preverbal message is mapped onto articulatory output during the language planning. Serial models predict a cascading sequence of hierarchically organized neural computations from conceptualization to articulation. In contrast, parallel models posit early simultaneous activation of multiple conceptual, phonological, and articulatory information in the language system. Here we asked whether such earliness is a distinctive property of the neural dynamics of word production. The combination of the millisecond precision of MEG with multivariate pattern analyses revealed subsequent onset times for the neural events supporting semantic and phonological/phonetic operations, progressing from posterior occipitotemporal to frontal sensor areas. The findings bring new insights for refining current theories of language production.

Stable maintenance of multiple representational formats in human visual short-term memory

Visual short-term memory (VSTM) enables humans to form a stable and coherent representation of the external world. However, the nature and temporal dynamics of the neural representations in VSTM that support this stability are barely understood. Here we combined human intracranial electroencephalography (iEEG) recordings with analyses using deep neural networks and semantic models to probe the representational format and temporal dynamics of information in VSTM. We found clear evidence that VSTM maintenance occurred in two distinct representational formats which originated from different encoding periods. The first format derived from an early encoding period (250 to 770 ms) corresponded to higher-order visual representations. The second format originated from a late encoding period (1,000 to 1,980 ms) and contained abstract semantic representations. These representational formats were overall stable during maintenance, with no consistent transformation across time. Nevertheless, maintenance of both representational formats showed substantial arrhythmic fluctuations, i.e., waxing and waning in irregular intervals. The increases of the maintained representational formats were specific to the phases of hippocampal low-frequency activity. Our results demonstrate that human VSTM simultaneously maintains representations at different levels of processing, from higher-order visual information to abstract semantic representations, which are stably maintained via coupling to hippocampal low-frequency activity.

Neural dynamics of perceptual inference and its reversal during imagery

After the presentation of a visual stimulus, neural processing cascades from low-level sensory areas to increasingly abstract representations in higher-level areas. It is often hypothesised that a reversal in neural processing underlies the generation of mental images as abstract representations are used to construct sensory representations in the absence of sensory input. According to predictive processing theories, such reversed processing also plays a central role in later stages of perception. Direct experimental evidence of reversals in neural information flow has been missing. Here, we used a combination of machine learning and magnetoencephalography to characterise neural dynamics in humans. We provide direct evidence for a reversal of the perceptual feed-forward cascade during imagery and show that, during perception, such reversals alternate with feed-forward processing in an 11 Hz oscillatory pattern. Together, these results show how common feedback processes support both veridical perception and mental imagery.

Differential temporal dynamics during visual imagery and perception

Visual perception and imagery rely on similar representations in the visual cortex. During perception, visual activity is characterized by distinct processing stages, but the temporal dynamics underlying imagery remain unclear. Here, we investigated the dynamics of visual imagery in human participants using magnetoencephalography. Firstly, we show that, compared to perception, imagery decoding becomes significant later and representations at the start of imagery already overlap with later time points. This suggests that during imagery, the entire visual representation is activated at once or that there are large differences in the timing of imagery between trials. Secondly, we found consistent overlap between imagery and perceptual processing around 160 ms and from 300 ms after stimulus onset. This indicates that the N170 gets reactivated during imagery and that imagery does not rely on early perceptual representations. Together, these results provide important insights for our understanding of the neural mechanisms of visual imagery.

On the Physiological Modulation and Potential Mechanisms Underlying Parieto-Occipital Alpha Oscillations

The parieto-occipital alpha (8–13 Hz) rhythm is by far the strongest spectral fingerprint in the human brain. Almost 90 years later, its physiological origin is still far from clear. In this Research Topic I review human pharmacological studies using electroencephalography (EEG) and magnetoencephalography (MEG) that investigated the physiological mechanisms behind posterior alpha. Based on results from classical and recent experimental studies, I find a wide spectrum of drugs that modulate parieto-occipital alpha power. Alpha frequency is rarely affected, but this might be due to the range of drug dosages employed. Animal and human pharmacological findings suggest that both GABA enhancers and NMDA blockers systematically decrease posterior alpha power. Surprisingly, most of the theoretical frameworks do not seem to embrace these empirical findings and the debate on the functional role of alpha oscillations has been polarized between the inhibition vs. active poles hypotheses. Here, I speculate that the functional role of alpha might depend on physiological excitation as much as on physiological inhibition. This is supported by animal and human pharmacological work showing that GABAergic, glutamatergic, cholinergic, and serotonergic receptors in the thalamus and the cortex play a key role in the regulation of alpha power and frequency. This myriad of physiological modulations fit with the view that the alpha rhythm is a complex rhythm with multiple sources supported by both thalamo-cortical and cortico-cortical loops. Finally, I briefly discuss how future research combining experimental measurements derived from theoretical predictions based of biophysically realistic computational models will be crucial to the reconciliation of these disparate findings.

Imaging the Spatiotemporal Dynamics of Cognitive Processes at High Temporal Resolution

This letter presents a noninvasive imaging technique that captures the exact timing and locations of cortical activity sequences that are specific to a cognitive process. These precise spatiotemporal sequences can be detected in the human brain as specific time-position pattern associated with a cognitive task. They are consistent with direct measurements of population activity recorded in nonhuman primates, thus suggesting that specific time-position patterns associated with a cognitive task can be identified. This imaging technique is based on estimating the amplitude of cortical current dipoles from MEG recordings. Although the spatial resolution of these estimations is poor (approximately 2 cm), the temporal resolution is high (milliseconds). We show that within these cortical current dipoles, time points of cortical activation can be identified as brief amplitude undulations and that sequences of these transients repeat with millisecond accuracy, hence making it possible to treat the timing of these transients as point processes. We illustrate the feasibility of finding spatiotemporal templates specific to the cognitive processes associated with following the rhythm of drumbeats that involve the activation at multiple cortical and cerebellar loci. These templates evolve at an accuracy of a few milliseconds. This approach can thus pave the way for new perspectives on the relationships between brain dynamics and cognition.

Concussion Alters the Functional Brain Processes of Visual Attention and Working Memory

Millions of North Americans sustain a concussion or a mild traumatic brain injury annually, and are at risk of cognitive, emotional, and physical sequelae. Although functional MRI (fMRI) studies have provided an initial framework for examining functional deficits induced by concussion, particularly working memory and attention, the temporal dynamics underlying these deficits are not well understood. We used magnetoencephalography (MEG), a modality with millisecond temporal resolution, in conjunction with a 1-back visual working memory (VWM) paradigm using scenes from everyday life to characterize spatiotemporal functional differences at specific VWM stages, in adults had had or had not had a recent concussion. MEG source-level differences between groups were determined by whole-brain analyses during encoding and recognition phases. Despite comparable behavioral performance, abnormal hypo- and hyperactivation patterns were found in brain areas involving frontoparietal, ventral occipitotemporal, temporal, and subcortical areas in concussed patients. These patterns and their timing varied as a function of VWM stagewise processing, linked to early attentional control, visuoperceptual scene processing, and VWM maintenance and retrieval processes. Parietal hypoactivation, starting at 60 ms during encoding, was correlated with symptom severity, possibly linked to impaired top-down attentional processing. Hyperactivation in the scene-selective occipitotemporal areas, the medial temporal complex, specifically the right hippocampus and orbitofrontal areas during encoding and/or recognition, lead us to posit inefficient but compensatory visuoperceptual, relational, and retrieval processing. Although injuries sustained after the concussion were considered "mild," these data suggest that they can have prolonged effects on early attentional and VWM processes.