Neural dynamics underlying target detection in the human brain

Correlates of implicit semantic processing as revealed by representational similarity analysis applied to EEG

Most researchers agree that some stages of object recognition can proceed implicitly. Implicit recognition occurs when an object is automatically and unintentionally encoded and represented in the brain even though the object is irrelevant to the current task. No consensus has been reached as to what level of semantic abstraction processing can go implicitly. An informative method to explore the level of abstraction and the time courses of informational content in neural representations is representational similarity analysis (RSA). Here, we apply RSA to EEG data recorded while participants processed semantics of visually presented objects. Explicit focus on semantics was given when participants classified images of objects as manmade or natural. For implicit processing of semantics, participants judged the location of images on the screen. The category animate/inanimate as well as more concrete categories (e.g., birds, fruit, musical instruments, etc.) are processed implicitly whereas the category manmade/natural is not processed implicitly.

Category-based attention facilitates memory search

We often need to decide whether the object we look at is also the object we look for. When we look for one specific object, this process can be facilitated by feature-based attention. However, when we look for many objects at the same time (e.g., the products on our shopping list) such a strategy may no longer be possible, as research has shown that we can actively prepare to detect only one or two objects at a time. Therefore, looking for multiple objects additionally requires long-term memory search, slowing down decision making. Interestingly, however, previous research has shown that distractor objects can be efficiently rejected during memory search when they are from a different category than the items in the memory set. Here, using EEG, we show that this efficiency is supported by top-down attention at the category level. In Experiment 1, human participants (both sexes) performed a memory search task on individually presented objects from different categories, most of which were distractors. We observed category-level attentional modulation of distractor processing from ∼150 ms after stimulus onset, expressed both as an evoked response modulation and as an increase in decoding accuracy of same-category distractors. In Experiment 2, memory search was performed on two concurrently presented objects. When both objects were distractors, spatial attention (indexed by the N2pc component) was directed to the object that was of the same category as the objects in the memory set. Together, these results demonstrate how top-down attention can facilitate memory search.Significance statement When we are in the supermarket, we repeatedly decide whether a product we look at (e.g., a banana) is on our memorized shopping list (e.g., apples, oranges, kiwis). This requires searching our memory, which takes time. However, when the product is of an entirely different category (e.g., dairy instead of fruit), the decision can be made quickly. Here, we used EEG to show that this between-category advantage in memory search tasks is supported by top-down attentional modulation of visual processing: The visual response evoked by distractor objects was modulated by category membership, and spatial attention was quickly directed to the location of within-category (vs. between-category) distractors. These results demonstrate a close link between attention and memory.

Early neural activity changes associated with stimulus detection during visual conscious perception

The earliest cortical neural signals following consciously perceived visual stimuli in humans are poorly understood. Using intracranial electroencephalography, we investigated neural activity changes associated with the earliest stages of stimulus detection during visual conscious perception. Participants (N = 10; 1,693 electrode contacts) completed a continuous performance task where subjects were asked to press a button when they saw a target letter among a series of nontargets. Broadband gamma power (40-115 Hz) was analyzed as marker of cortical population neural activity. Regardless of target or nontarget letter type, we observed early gamma power changes within 30-180 ms from stimulus onset in a network including increases in bilateral occipital, fusiform, frontal (including frontal eye fields), and medial temporal cortex; increases in left lateral parietal-temporal cortex; and decreases in the right anterior medial occipital cortex. No significant differences were observed between target and nontarget stimuli until >180 ms post-stimulus, when we saw greater gamma power increases in left motor and premotor areas, suggesting a possible role in perceptual decision-making and/or motor responses with the right hand. The early gamma power findings support a broadly distributed cortical visual detection network that is engaged at early times tens of milliseconds after signal transduction from the retina.

Hemodynamic functional connectivity optimization of frequency EEG microstates enables attention LSTM framework to classify distinct temporal cortical communications of different cognitive tasks

Temporal analysis of global cortical communication of cognitive tasks in coarse EEG information is still challenging due to the underlying complex neural mechanisms. This study proposes an attention-based time-series deep learning framework that processes fMRI functional connectivity optimized quasi-stable frequency microstates for classifying distinct temporal cortical communications of the cognitive task. Seventy volunteers were subjected to visual target detection tasks, and their electroencephalogram (EEG) and functional MRI (fMRI) were acquired simultaneously. At first, the acquired EEG information was preprocessed and bandpass to delta, theta, alpha, beta, and gamma bands and then subjected to quasi-stable frequency-microstate estimation. Subsequently, time-series elicitation of each frequency microstates is optimized with graph theory measures of simultaneously eliciting fMRI functional connectivity between frontal, parietal, and temporal cortices. The distinct neural mechanisms associated with each optimized frequency-microstate were analyzed using microstate-informed fMRI. Finally, these optimized, quasi-stable frequency microstates were employed to train and validate the attention-based Long Short-Term Memory (LSTM) time-series architecture for classifying distinct temporal cortical communications of the target from other cognitive tasks. The temporal, sliding input sampling windows were chosen between 180 to 750 ms/segment based on the stability of transition probabilities of the optimized microstates. The results revealed 12 distinct frequency microstates capable of deciphering target detections' temporal cortical communications from other task engagements. Particularly, fMRI functional connectivity measures of target engagement were observed significantly correlated with the right-diagonal delta (r = 0.31), anterior-posterior theta (r = 0.35), left-right theta (r = - 0.32), alpha (r = - 0.31) microstates. Further, neuro-vascular information of microstate-informed fMRI analysis revealed the association of delta/theta and alpha/beta microstates with cortical communications and local neural processing, respectively. The classification accuracies of the attention-based LSTM were higher than the traditional LSTM architectures, particularly the frameworks that sampled the EEG data with a temporal width of 300 ms/segment. In conclusion, the study demonstrates reliable temporal classifications of global cortical communication of distinct tasks using an attention-based LSTM utilizing fMRI functional connectivity optimized quasi-stable frequency microstates.

Mesoscopic physiological interactions in the human brain reveal small-world properties

Cognition depends on rapid and robust communication between neural circuits spanning different brain areas. We investigated the mesoscopic network of cortico-cortical interactions in the human brain in an extensive dataset consisting of 6,024 h of intracranial field potential recordings from 4,142 electrodes in 48 subjects. We evaluated communication between brain areas at the network level across different frequency bands. The interaction networks were validated against known anatomical measurements and neurophysiological interactions in humans and monkeys. The resulting human brain interactome is characterized by a broad and spatially specific, dynamic, and extensive network. The physiological interactome reveals small-world properties, which we conjecture might facilitate efficient and reliable information transmission. The interaction dynamics correlate with the brain sleep/awake state. These results constitute initial steps toward understanding how the interactome orchestrates cortical communication and provide a reference for future efforts assessing how dysfunctional interactions may lead to mental disorders.

Cognition relies on rapid and robust communication between brain areas. Wang et al. leverage multi-day intracranial field potential recordings to characterize the human mesoscopic functional interactome. They validated the methods using monkey anatomical and physiological data. The human interactome reveals small-world properties and is modulated by sleep versus awake state.

Dissociating conscious and unconscious influences on visual detection effects

The scope of unconscious processing is highly debated, with recent studies showing that even high-level functions such as perceptual integration and category-based attention occur unconsciously. For example, upright faces that are suppressed from awareness through interocular suppression break into awareness more quickly than inverted faces. Similarly, verbal object cues boost otherwise invisible objects into awareness. Here, we replicate these findings, but find that they reflect a general difference in detectability not specific to interocular suppression. To dissociate conscious and unconscious influences on visual detection effects, we use an additional discrimination task to rule out conscious processes as a cause for these differences. Results from this detection-discrimination dissociation paradigm reveal that, while face orientation is processed unconsciously, category-based attention requires awareness. These findings provide insights into the function of conscious perception and offer an experimental approach for mapping out the scope and limits of unconscious processing.

Eye tracking metrics to screen and assess cognitive impairment in patients with neurological disorders

PURPOSE OF REVIEW

Eye tracking is a powerful method to investigate the relationship between behavior and neural mechanisms. In recent years, eye movement analysis has been used in patients with neurological disorders to assess cognitive function. In this review, we explore the latest eye tracking researches in neurological disorders that are commonly associated with cognitive deficits, specifically, amyotrophic lateral sclerosis (ALS), Alzheimer's disease (AD), Parkinson's disease (PD), multiple sclerosis (MS), and epilepsy. We focus on the application of ocular measures in these disorders, with the goal of understanding how eye tracking technology can be used in the clinical setting.

FINDINGS

Eye tracking tasks (especially saccadic tasks) are often used as an adjunct to traditional scales for cognitive assessment. Eye tracking data confirmed that executive dysfunction is common in PD and ALS, whereas AD and MS are characterized by attention deficits. Research in evaluating cognitive function in epilepsy using eye tracking is still in its early stages, but this approach has shown advantages as a sensitive quantitative method with high temporal and spatial resolution. Eye tracking technology can facilitate the assessment of cognitive impairment with higher temporal resolution and finer granularity than traditional cognitive assessment. Oculomotor data collected during cognitive tasks can provide insight into biological processes. Eye tracking provides a nonverbal and less cognitively demanding method of measuring disease progression in cognitively impaired patients.

Differential Sources for 2 Neural Signatures of Target Detection: An Electrocorticography Study

Electrophysiology and neuroimaging provide conflicting evidence for the neural contributions to target detection. Scalp electroencephalography (EEG) studies localize the P3b event-related potential component mainly to parietal cortex, whereas neuroimaging studies report activations in both frontal and parietal cortices. We addressed this discrepancy by examining the sources that generate the target-detection process using electrocorticography (ECoG). We recorded ECoG activity from cortex in 14 patients undergoing epilepsy monitoring, as they performed an auditory or visual target-detection task. We examined target-related responses in 2 domains: high frequency band (HFB) activity and the P3b. Across tasks, we observed a greater proportion of electrodes that showed target-specific HFB power relative to P3b over frontal cortex, but their proportions over parietal cortex were comparable. Notably, there was minimal overlap in the electrodes that showed target-specific HFB and P3b activity. These results revealed that the target-detection process is characterized by at least 2 different neural markers with distinct cortical distributions. Our findings suggest that separate neural mechanisms are driving the differential patterns of activity observed in scalp EEG and neuroimaging studies, with the P3b reflecting EEG findings and HFB activity reflecting neuroimaging findings, highlighting the notion that target detection is not a unitary phenomenon.

The Neural Dynamics of Attentional Selection in Natural Scenes

The human visual system can only represent a small subset of the many objects present in cluttered scenes at any given time, such that objects compete for representation. Despite these processing limitations, the detection of object categories in cluttered natural scenes is remarkably rapid. How does the brain efficiently select goal-relevant objects from cluttered scenes? In the present study, we used multivariate decoding of magneto-encephalography (MEG) data to track the neural representation of within-scene objects as a function of top-down attentional set. Participants detected categorical targets (cars or people) in natural scenes. The presence of these categories within a scene was decoded from MEG sensor patterns by training linear classifiers on differentiating cars and people in isolation and testing these classifiers on scenes containing one of the two categories. The presence of a specific category in a scene could be reliably decoded from MEG response patterns as early as 160 ms, despite substantial scene clutter and variation in the visual appearance of each category. Strikingly, we find that these early categorical representations fully depend on the match between visual input and top-down attentional set: only objects that matched the current attentional set were processed to the category level within the first 200 ms after scene onset. A sensor-space searchlight analysis revealed that this early attention bias was localized to lateral occipitotemporal cortex, reflecting top-down modulation of visual processing. These results show that attention quickly resolves competition between objects in cluttered natural scenes, allowing for the rapid neural representation of goal-relevant objects.

SIGNIFICANCE STATEMENT

Efficient attentional selection is crucial in many everyday situations. For example, when driving a car, we need to quickly detect obstacles, such as pedestrians crossing the street, while ignoring irrelevant objects. How can humans efficiently perform such tasks, given the multitude of objects contained in real-world scenes? Here we used multivariate decoding of magnetoencephalogaphy data to characterize the neural underpinnings of attentional selection in natural scenes with high temporal precision. We show that brain activity quickly tracks the presence of objects in scenes, but crucially only for those objects that were immediately relevant for the participant. These results provide evidence for fast and efficient attentional selection that mediates the rapid detection of goal-relevant objects in real-world environments.

A null model for cortical representations with grandmothers galore

There has been extensive discussion in the literature about the extent to which cortical representations can be described as localist or distributed. Here we discuss a simple null model that encompasses a family of related architectures describing the transformation of signals throughout the parts of the visual system involved in object recognition. This family of models constitutes a rigorous first approximation to explain the neurophysiological properties of ventral visual cortex. This null model contains both distributed and local representations throughout the entire hierarchy of computations and the responses of individual units are meaningful and interpretable when encoding is adequately defined for each computational stage.

Network Anisotropy Trumps Noise for Efficient Object Coding in Macaque Inferior Temporal Cortex

How neuronal ensembles compute information is actively studied in early visual cortex. Much less is known about how local ensembles function in inferior temporal (IT) cortex, the last stage of the ventral visual pathway that supports visual recognition. Previous reports suggested that nearby neurons carry information mostly independently, supporting efficient processing (Barlow, 1961). However, others postulate that noise covariation effects may depend on network anisotropy/homogeneity and on how the covariation relates to representation. Do slow trial-by-trial noise covariations increase or decrease IT's object coding capability, how does encoding capability relate to correlational structure (i.e., the spatial pattern of signal and noise redundancy/homogeneity across neurons), and does knowledge of correlational structure matter for decoding? We recorded simultaneously from ∼80 spiking neurons in ∼1 mm(3) of macaque IT under light neurolept anesthesia. Noise correlations were stronger for neurons with correlated tuning, and noise covariations reduced object encoding capability, including generalization across object pose and illumination. Knowledge of noise covariations did not lead to better decoding performance. However, knowledge of anisotropy/homogeneity improved encoding and decoding efficiency by reducing the number of neurons needed to reach a given performance level. Such correlated neurons were found mostly in supragranular and infragranular layers, supporting theories that link recurrent circuitry to manifold representation. These results suggest that redundancy benefits manifold learning of complex high-dimensional information and that subsets of neurons may be more immune to noise covariation than others.

SIGNIFICANCE STATEMENT

How noise affects neuronal population coding is poorly understood. By sampling densely from local populations supporting visual object recognition, we show that recurrent circuitry supports useful representations and that subsets of neurons may be more immune to noise covariation than others.

Preparatory Activity in Posterior Temporal Cortex Causally Contributes to Object Detection in Scenes

Theories of visual selective attention propose that top-down preparatory attention signals mediate the selection of task-relevant information in cluttered scenes. Neuroimaging and electrophysiology studies have provided correlative evidence for this hypothesis, finding increased activity in target-selective neural populations in visual cortex in the period between a search cue and target onset. In this study, we used online TMS to test whether preparatory neural activity in visual cortex is causally involved in naturalistic object detection. In two experiments, participants detected the presence of object categories (cars, people) in a diverse set of photographs of real-world scenes. TMS was applied over a region in posterior temporal cortex identified by fMRI as carrying category-specific preparatory activity patterns. Results showed that TMS applied over posterior temporal cortex before scene onset (-200 and -100 msec) impaired the detection of object categories in subsequently presented scenes, relative to vertex and early visual cortex stimulation. This effect was specific to category level detection and was related to the type of attentional template participants adopted, with the strongest effects observed in participants adopting category level templates. These results provide evidence for a causal role of preparatory attention in mediating the detection of objects in cluttered daily-life environments.

Correlated activity supports efficient cortical processing

Visual recognition is a computational challenge that is thought to occur via efficient coding. An important concept is sparseness, a measure of coding efficiency. The prevailing view is that sparseness supports efficiency by minimizing redundancy and correlations in spiking populations. Yet, we recently reported that "choristers", neurons that behave more similarly (have correlated stimulus preferences and spontaneous coincident spiking), carry more generalizable object information than uncorrelated neurons ("soloists") in macaque inferior temporal (IT) cortex. The rarity of choristers (as low as 6% of IT neurons) indicates that they were likely missed in previous studies. Here, we report that correlation strength is distinct from sparseness (choristers are not simply broadly tuned neurons), that choristers are located in non-granular output layers, and that correlated activity predicts human visual search efficiency. These counterintuitive results suggest that a redundant correlational structure supports efficient processing and behavior.

Spatiotemporal dynamics of affective picture processing revealed by intracranial high-gamma modulations

Our comprehension of the neural mechanisms underlying emotional information processing has largely benefited from noninvasive electrophysiological and functional neuroimaging techniques in recent years. However, the spatiotemporal dynamics of the neural events occurring during emotional processing remain imprecise due to the limited combination of spatial and temporal resolution provided by these techniques. This study examines the modulations of high-frequency activity of intracranial electroencephalography recordings associated with affective picture valence, in epileptic patients awaiting neurosurgery. Recordings were obtained from subdural grids and depth electrodes in eight patients while they viewed a series of unpleasant, pleasant and neutral pictures from the International Affective Picture System. Broadband high-gamma (70-150 Hz) power was computed for separate 100-ms time windows and compared according to ratings of emotional valence. Compared to emotionally neutral or pleasant pictures, unpleasant stimuli were associated with an early and long-lasting (≈200-1,000 ms) bilateral increase in high-gamma activity in visual areas of the occipital and temporal lobes, together with a late and transient (≈500-800 ms) decrease found bilaterally in the lateral prefrontal cortex (PFC). Pleasant pictures were associated with increased gamma activity in the occipital cortex, compared to the emotionally neutral stimuli. Consistent with previous studies, our results provide direct evidence of emotion-related modulations in the visual ventral pathway during picture processing. Results in the lateral PFC also shed light on the neural mechanisms underlying its role in negative emotions processing. This study demonstrates the utility of intracranial high-gamma modulations to study emotional process with a high spatiotemporal precision.

Differential activation of human core, non-core and auditory-related cortex during speech categorization tasks as revealed by intracranial recordings

Speech perception requires that sounds be transformed into speech-related objects with lexical and semantic meaning. It is unclear at what level in the auditory pathways this transformation emerges. Primary auditory cortex has been implicated in both representation of acoustic sound attributes and sound objects. While non-primary auditory cortex located on the posterolateral superior temporal gyrus (PLST) is clearly involved in acoustic-to-phonetic pre-lexical representations, it is unclear what role this region plays in auditory object formation. Additional data support the importance of prefrontal cortex in the formation of auditory objects, while other data would implicate this region in auditory object selection. To help clarify the respective roles of auditory and auditory-related cortex in the formation and selection of auditory objects, we examined high gamma activity simultaneously recorded directly from Heschl's gyrus (HG), PLST and prefrontal cortex, while subjects performed auditory semantic detection tasks. Subjects were patients undergoing evaluation for treatment of medically intractable epilepsy. We found that activity in posteromedial HG and early activity on PLST was robust to sound stimuli regardless of their context, and minimally modulated by tasks. Later activity on PLST could be strongly modulated by semantic context, but not by behavioral performance. Activity within prefrontal cortex also was related to semantic context, and did co-vary with behavior. We propose that activity in posteromedial HG and early activity on PLST primarily reflect the representation of spectrotemporal sound attributes. Later activity on PLST represents a pre-lexical processing stage and is an intermediate step in the formation of word objects. Activity in prefrontal cortex appears directly involved in word object selection. The roles of other auditory and auditory-related cortical areas in the formation of word objects remain to be explored.