Attention to Multiple Objects Facilitates Their Integration in Prefrontal and Parietal Cortex

Intermodulation frequency components in steady-state visual evoked potentials: Generation, characteristics and applications

The steady-state visual evoked potentials (SSVEPs), evoked by dual-frequency or multi-frequency stimulation, likely contains intermodulation frequency components (IMs). Visual IMs are products of nonlinear integration of neural signals and can be evoked by various paradigms that induce neural interaction. IMs have demonstrated many interesting and important characteristics in cognitive psychology, clinical neuroscience, brain-computer interface and other fields, and possess substantial research potential. In this paper, we first review the definition of IMs and summarize the stimulation paradigms capable of inducing them, along with the possible neural origins of IMs. Subsequently, we describe the characteristics and derived applications of IMs in previous studies, and then introduced three signal processing methods favored by researchers to enhance the signal-to-noise ratio of IMs. Finally, we summarize the characteristics of IMs, and propose several potential future research directions related to IMs.

Measuring attentional selection of object categories using hierarchical frequency tagging

In the present study, we used Hierarchical Frequency Tagging (Gordon et al., 2017) to investigate in electroencephalography how different levels of the neural processing hierarchy interact with category-selective attention during visual object recognition. We constructed stimulus sequences of cyclic wavelet scrambled face and house stimuli at two different frequencies (f1 = 0.8 Hz and f2 = 1 Hz). For each trial, two stimulus sequences of different frequencies were superimposed and additionally augmented by a sinusoidal contrast modulation with f3 = 12.5 Hz. This allowed us to simultaneously assess higher level processing using semantic wavelet-induced frequency-tagging (SWIFT) and processing in earlier visual levels using steady-state visually evoked potentials (SSVEPs), along with their intermodulation (IM) components. To investigate the category specificity of the SWIFT signal, we manipulated the category congruence between target and distractor by superimposing two sequences containing stimuli from the same or different object categories. Participants attended to one stimulus (target) and ignored the other (distractor). Our results showed successful tagging of different levels of the cortical hierarchy. Using linear mixed-effects modeling, we detected different attentional modulation effects on lower versus higher processing levels. SWIFT and IM components were substantially increased for target versus distractor stimuli, reflecting attentional selection of the target stimuli. In addition, distractor stimuli from the same category as targets elicited stronger SWIFT signals than distractor stimuli from a different category indicating category-selective attention. In contrast, for IM components, this category-selective attention effect was largely absent, indicating that IM components probably reflect more stimulus-specific processing.

Spatial and temporal (non)binding of audiovisual rhythms in sensorimotor synchronisation

Human movement synchronisation with moving objects strongly relies on visual input. However, auditory information also plays an important role, since real environments are intrinsically multimodal. We used electroencephalography (EEG) frequency tagging to investigate the selective neural processing and integration of visual and auditory information during motor tracking and tested the effects of spatial and temporal congruency between audiovisual modalities. EEG was recorded while participants tracked with their index finger a red flickering (rate f_V = 15 Hz) dot oscillating horizontally on a screen. The simultaneous auditory stimulus was modulated in pitch (rate f_A = 32 Hz) and lateralised between left and right audio channels to induce perception of a periodic displacement of the sound source. Audiovisual congruency was manipulated in terms of space in Experiment 1 (no motion, same direction or opposite direction), and timing in Experiment 2 (no delay, medium delay or large delay). For both experiments, significant EEG responses were elicited at f_V and f_A tagging frequencies. It was also hypothesised that intermodulation products corresponding to the nonlinear integration of visual and auditory stimuli at frequencies f_V ± f_A would be elicited, due to audiovisual integration, especially in Congruent conditions_. However, these components were not observed. Moreover, synchronisation and EEG results were not influenced by congruency manipulations, which invites further exploration of the conditions which may modulate audiovisual processing and the motor tracking of moving objects.

Decoding semantics from intermodulation responses in frequency-tagged stereotactic EEG

BACKGROUND

Humans perform object recognition using holistic processing, which is different from computers. Intermodulation responses in the steady-state visual evoked potential (SSVEP) of scalp electroencephalography (EEG) have recently been used as an objective label for holistic processing.

NEW METHOD

Using stereotactic EEG (sEEG) to record SSVEP directly from inside of the brain, we aimed to decode Chinese characters from non-characters with activation from multiple brain areas including occipital, parietal, temporal, and frontal cortices.

RESULTS

Semantic categories could be decoded from responses at the intermodulation frequency with high accuracy (80%-90%), but not the base frequency. Moreover, semantic categories could be decoded with activation from multiple areas including temporal, parietal, and frontal areas.

COMPARISON WITH EXISTING METHOD(S)

Previous studies investigated holistic processing in faces and words with frequency-tagged scalp EEGs. The current study extended the results to stereotactic EEG signals directly recorded from the brain.

CONCLUSIONS

The human brain applies holistic processing in recognizing objects like Chinese characters. Our findings could be extended to an add-on feature in the existing SSVEP BCI speller.

Neural decoding dissociates perceptual grouping between proximity and similarity in visual perception

Unlike single grouping principle, cognitive neural mechanism underlying the dissociation across two or more grouping principles is still unclear. In this study, a dimotif lattice paradigm that can adjust the strength of one grouping principle was used to inspect how, when, and where the processing of two grouping principles (proximity and similarity) were carried out in human brain. Our psychophysical findings demonstrated that similarity grouping effect was enhanced with reduced proximity effect when the grouping cues of proximity and similarity were presented simultaneously. Meanwhile, EEG decoding was performed to reveal the specific cognitive patterns involved in each principle by using time-resolved MVPA. More importantly, the onsets of dissociation between 2 grouping principles coincided within 3 time windows: the early-stage proximity-defined local visual element arrangement in middle occipital cortex, the middle-stage processing for feature selection modulating low-level visual cortex such as inferior occipital cortex and fusiform cortex, and the high-level cognitive integration to make decisions for specific grouping preference in the parietal areas. In addition, it was discovered that the brain responses were highly correlated with behavioral grouping. Therefore, our study provides direct evidence for a link between the human perceptual space of grouping decision-making and neural space of brain activation patterns.

Simultaneous time processing in children and adults: When attention predicts temporal interference effects

Children from 5 to 8 years of age, as well as adults, performed a temporal reproduction task in both a solo-timing condition and a multi-timing condition, with different durations presented simultaneously. In the multi-timing condition, all durations were processed because the participants did not know in advance which stimulus needed to be judged. In a first experiment, two or three durations were presented with a synchrony of their onset. In a second experiment, two durations were presented simultaneously with asynchrony of their offset, different lengths of the concurrent duration, and different presentation orders. In addition, the participants' cognitive abilities in terms of selective attention, as well as short-term and working memory, were assessed with different neuropsychological tests. The results of both experiments showed that children and adults alike were able to process multiple durations simultaneously. However, the simultaneous presentation of different durations generated a temporal interference effect in children and adults, resulting in longer and more variable time estimates. This temporal interference effect was nevertheless higher in children due to their limited attention capacities. Therefore, a developmental improvement in the ability to process different durations simultaneously is related to the cognitive development of attention capacities.

Steady state evoked potential (SSEP) responses in the primary and secondary somatosensory cortices of anesthetized cats: Nonlinearity characterized by harmonic and intermodulation frequencies

When presented with an oscillatory sensory input at a particular frequency, F [Hz], neural systems respond with the corresponding frequency, f [Hz], and its multiples. When the input includes two frequencies (F1 and F2) and they are nonlinearly integrated in the system, responses at intermodulation frequencies (i.e., n1*f1+n2*f2 [Hz], where n1 and n2 are non-zero integers) emerge. Utilizing these properties, the steady state evoked potential (SSEP) paradigm allows us to characterize linear and nonlinear neural computation performed in cortical neurocircuitry. Here, we analyzed the steady state evoked local field potentials (LFPs) recorded from the primary (S1) and secondary (S2) somatosensory cortex of anesthetized cats (maintained with alfaxalone) while we presented slow (F1 = 23Hz) and fast (F2 = 200Hz) somatosensory vibration to the contralateral paw pads and digits. Over 9 experimental sessions, we recorded LFPs from N = 1620 and N = 1008 bipolar-referenced sites in S1 and S2 using electrode arrays. Power spectral analyses revealed strong responses at 1) the fundamental (f1, f2), 2) its harmonic, 3) the intermodulation frequencies, and 4) broadband frequencies (50-150Hz). To compare the computational architecture in S1 and S2, we employed simple computational modeling. Our modeling results necessitate nonlinear computation to explain SSEP in S2 more than S1. Combined with our current analysis of LFPs, our paradigm offers a rare opportunity to constrain the computational architecture of hierarchical organization of S1 and S2 and to reveal how a large-scale SSEP can emerge from local neural population activities.

Associations between brain-derived neurotrophic factor and cognitive impairment in panic disorder

INTRODUCTION

Our study was designed to examine the relationship between Brain-Derived Neurotrophic Factor (BDNF) genotypes (rs6265, Val66Met), BDNF plasma levels, and cognitive impairment in Chinese patients with panic disorder (PD).

METHODS

Total 85 patients with PD and 91 healthy controls finally completed all assessments. The severity of panic symptoms and whole anxiety of PD was measured by Panic Disorder Severity Scale-Chinese Version (PDSS-CV) and Hamilton Anxiety Scale (HAMA-14). Montreal Cognitive Assessment (MoCA) and some neurocognitive measures were conducted to evaluate the cognitive performance. All participants were detected for the plasma BDNF levels and BDNF Val66Met polymorphism before assessment and treatment.

RESULTS

No significant differences were found in the BDNF allele frequencies and the BDNF genotype distributions between healthy controls and PD patients. BDNF Met/Met genotype was associated with lower BDNF plasma levels in PD patients, and PD patients with BDNF Met/Met genotype had the lower scores in the attention and speed of processing domains compared to those with Val/Val and Met/Val genotype (p's < .05). Among PD patients, the BDNF plasma levels showed moderate positive correlations with Stroop interference (r = .60, p < .001). Using the MoCA data, the BDNF plasma levels were correlated with delayed memory (r = .50, p < .001), verbal learning (r = .45, p < .001), and total scores of MoCA (r = .51, p < .001).

CONCLUSIONS

The BDNF Met/Met genotype may be associated with lower BDNF plasma levels and cognitive impairments in PD patients.

The SSVEP tracks attention, not consciousness, during perceptual filling-in

Research on the neural basis of conscious perception has almost exclusively shown that becoming aware of a stimulus leads to increased neural responses. By designing a novel form of perceptual filling-in (PFI) overlaid with a dynamic texture display, we frequency-tagged multiple disappearing targets as well as their surroundings. We show that in a PFI paradigm, the disappearance of a stimulus and subjective invisibility is associated with increases in neural activity, as measured with steady-state visually evoked potentials (SSVEPs), in electroencephalography (EEG). We also find that this increase correlates with alpha-band activity, a well-established neural measure of attention. These findings cast doubt on the direct relationship previously reported between the strength of neural activity and conscious perception, at least when measured with current tools, such as the SSVEP. Instead, we conclude that SSVEP strength more closely measures changes in attention.

Electrophysiological correlates of gist perception: a steady-state visually evoked potentials study

Gist perception refers to perceiving the substance or general meaning of a scene. To investigate its neuronal mechanisms, we used the steady-state visually evoked potential (SSVEP) method—an evoked oscillatory cortical response at the same frequency as a visual stimulus flickered at this frequency. Two neighboring stimuli were flickered at different frequencies f₁ and f₂, for example, a drawing of a sun on the left side of the screen flickering at 8.6 Hz and the drawing of a parasol on the right side of the screen flickering at 12 Hz. SSVEPs enabled us to separate the responses to the two distinct stimuli by extracting oscillatory brain responses at f₁ and f₂. Additionally, it allowed to investigate intermodulation frequencies, that is, the brain’s response at a linear combination of f₁ and f₂ (here at f₁ + f₂ = 20.6 Hz) as an indicator of processing shared aspects of the input, that is, gist perception (here: a beach scene). We recorded high-density EEG of 18 participants. Results revealed clear and separable neuronal oscillations at f₁ and f₂. Additionally, occipital electrodes showed increased amplitudes at the intermodulation frequency in related as compared to unrelated pairs. The increase in intermodulation frequency was associated with bilateral temporal and parietal lobe activation, probably reflecting the interaction of local object representations as a basis for activating the gist network. The study demonstrates that SSVEPs are an excellent method to unravel mechanisms underlying the processing within multi-stimulus displays in the context of gist perception.

When Conflict Cannot be Avoided: Relative Contributions of Early Selection and Frontal Executive Control in Mitigating Stroop Conflict

When viewing familiar stimuli (e.g., common words), processing is highly automatized such that it can interfere with the processing of incompatible sensory information. At least two mechanisms may help mitigate this interference. Early selection accounts posit that attentional processes filter out distracting sensory information to avoid conflict. Alternatively, late selection accounts hold that all sensory inputs receive full semantic analysis and that frontal executive mechanisms are recruited to resolve conflict. To test how these mechanisms operate to overcome conflict induced by highly automatized processing, we developed a novel version of the color-word Stroop task, where targets and distractors were simultaneously flickered at different frequencies. We measured the quality of early sensory processing by assessing the amplitude of steady-state visually evoked potentials (SSVEPs) elicited by targets and distractors. We also indexed frontal executive processes by assessing changes in frontal theta oscillations induced by color-word incongruency. We found that target- and distractor-related SSVEPs were not modulated by changes in the level of conflict whereas frontal theta activity increased on high compared to low conflict trials. These results suggest that frontal executive processes play a more dominant role in mitigating cognitive interference driven by the automatic tendency to process highly familiar stimuli.

Spatial attention enhances cortical tracking of quasi-rhythmic visual stimuli

Successfully interpreting and navigating our natural visual environment requires us to track its dynamics constantly. Additionally, we focus our attention on behaviorally relevant stimuli to enhance their neural processing. Little is known, however, about how sustained attention affects the ongoing tracking of stimuli with rich natural temporal dynamics. Here, we used MRI-informed source reconstructions of magnetoencephalography (MEG) data to map to what extent various cortical areas track concurrent continuous quasi-rhythmic visual stimulation. Further, we tested how top-down visuo-spatial attention influences this tracking process. Our bilaterally presented quasi-rhythmic stimuli covered a dynamic range of 4-20 ​Hz, subdivided into three distinct bands. As an experimental control, we also included strictly rhythmic stimulation (10 vs 12 ​Hz). Using a spectral measure of brain-stimulus coupling, we were able to track the neural processing of left vs. right stimuli independently, even while fluctuating within the same frequency range. The fidelity of neural tracking depended on the stimulation frequencies, decreasing for higher frequency bands. Both attended and non-attended stimuli were tracked beyond early visual cortices, in ventral and dorsal streams depending on the stimulus frequency. In general, tracking improved with the deployment of visuo-spatial attention to the stimulus location. Our results provide new insights into how human visual cortices process concurrent dynamic stimuli and provide a potential mechanism - namely increasing the temporal precision of tracking - for boosting the neural representation of attended input.

Domain-general and domain-specific computations in single word processing

Language comprehension relies on a multitude of domain-general and domain-specific cognitive operations. This study asks whether the domain-specific grammatical computations are obligatorily invoked whenever we process linguistic inputs. Using fMRI and three complementary measures of neural activity, we tested how domain-general and domain-specific demands of single word comprehension engage cortical language networks, and whether the left frontotemporal network (commonly taken to support domain-specific grammatical computations) automatically processes grammatical information present in inflectionally complex words. In a natural listening task, participants were presented with words that manipulated domain-general and domain-specific processing demands in a 2 × 2 manner. The results showed that only domain-general demands of mapping words onto their representations consistently engaged the language processing system during single word comprehension, triggering increased activity and connectivity in bilateral frontotemporal regions, as well as bilateral encoding across multivoxel activity patterns. In contrast, inflectional complexity failed to activate left frontotemporal regions in this task, implying that domain-specific grammatical processing in the left hemisphere is not automatically triggered when the processing context does not specifically require such analysis. This suggests that cortical computations invoked by language processing critically depend on the current communicative goals and demands, underlining the importance of domain-general processes in language comprehension, and arguing against the strong domain-specific view of the LH network function.

From intermodulation components to visual perception and cognition-a review

Perception results from complex interactions among sensory and cognitive processes across hierarchical levels in the brain. Intermodulation (IM) components, used in frequency tagging neuroimaging designs, have emerged as a promising direct measure of such neural interactions. IMs have initially been used in electroencephalography (EEG) to investigate low-level visual processing. In a more recent trend, IMs in EEG and other neuroimaging methods are being used to shed light on mechanisms of mid- and high-level perceptual processes, including the involvement of cognitive functions such as attention and expectation. Here, we provide an account of various mechanisms that may give rise to IMs in neuroimaging data, and what these IMs may look like. We discuss methodologies that can be implemented for different uses of IMs and we demonstrate how IMs can provide insights into the existence, the degree and the type of neural integration mechanisms at hand. We then review a range of recent studies exploiting IMs in visual perception research, placing an emphasis on high-level vision and the influence of awareness and cognition on visual processing. We conclude by suggesting future directions that can enhance the benefits of IM-methodology in perception research.

Attentional Weighting in the Face Processing Network: A Magnetic Response Image-guided Magnetoencephalography Study Using Multiple Cyclic Entrainments

We recorded magnetoencephalography using a neural entrainment paradigm with compound face stimuli that allowed for entraining the processing of various parts of a face (eyes, mouth) as well as changes in facial identity. Our magnetic response image-guided magnetoencephalography analyses revealed that different subnodes of the human face processing network were entrained differentially according to their functional specialization. Whereas the occipital face area was most responsive to the rate at which face parts (e.g., the mouth) changed, and face patches in the STS were mostly entrained by rhythmic changes in the eye region, the fusiform face area was the only subregion that was strongly entrained by the rhythmic changes in facial identity. Furthermore, top-down attention to the mouth, eyes, or identity of the face selectively modulated the neural processing in the respective area (i.e., occipital face area, STS, or fusiform face area), resembling behavioral cue validity effects observed in the participants' RT and detection rate data. Our results show the attentional weighting of the visual processing of different aspects and dimensions of a single face object, at various stages of the involved visual processing hierarchy.

Object-based attention in complex, naturalistic auditory streams

In vision, objects have been described as the 'units' on which non-spatial attention operates in many natural settings. Here, we test the idea of object-based attention in the auditory domain within ecologically valid auditory scenes, composed of two spatially and temporally overlapping sound streams (speech signal vs. environmental soundscapes in Experiment 1 and two speech signals in Experiment 2). Top-down attention was directed to one or the other auditory stream by a non-spatial cue. To test for high-level, object-based attention effects we introduce an auditory repetition detection task in which participants have to detect brief repetitions of auditory objects, ruling out any possible confounds with spatial or feature-based attention. The participants' responses were significantly faster and more accurate in the valid cue condition compared to the invalid cue condition, indicating a robust cue-validity effect of high-level, object-based auditory attention.