Electrophysiological correlates of categorization: P300 amplitude as index of target similarity

A comparison of visual and auditory EEG interfaces for robot multi-stage task control

Shared autonomy holds promise for assistive robotics, whereby physically-impaired people can direct robots to perform various tasks for them. However, a robot that is capable of many tasks also introduces many choices for the user, such as which object or location should be the target of interaction. In the context of non-invasive brain-computer interfaces for shared autonomy-most commonly electroencephalography-based-the two most common choices are to provide either auditory or visual stimuli to the user-each with their respective pros and cons. Using the oddball paradigm, we designed comparable auditory and visual interfaces to speak/display the choices to the user, and had users complete a multi-stage robotic manipulation task involving location and object selection. Users displayed differing competencies-and preferences-for the different interfaces, highlighting the importance of considering modalities outside of vision when constructing human-robot interfaces.

Decoding neural activity to assess individual latent state in ecologically valid contexts

Objective.Currently, there exists very few ways to isolate cognitive processes, historically defined via highly controlled laboratory studies, in more ecologically valid contexts. Specifically, it remains unclear as to what extent patterns of neural activity observed under such constraints actually manifest outside the laboratory in a manner that can be used to make accurate inferences about latent states, associated cognitive processes, or proximal behavior. Improving our understanding of when and how specific patterns of neural activity manifest in ecologically valid scenarios would provide validation for laboratory-based approaches that study similar neural phenomena in isolation and meaningful insight into the latent states that occur during complex tasks.Approach.Domain generalization methods, borrowed from the work of the brain-computer interface community, have the potential to capture high-dimensional patterns of neural activity in a way that can be reliably applied across experimental datasets in order to address this specific challenge. We previously used such an approach to decode phasic neural responses associated with visual target discrimination. Here, we extend that work to more tonic phenomena such as internal latent states. We use data from two highly controlled laboratory paradigms to train two separate domain-generalized models. We apply the trained models to an ecologically valid paradigm in which participants performed multiple, concurrent driving-related tasks while perched atop a six-degrees-of-freedom ride-motion simulator.Main Results.Using the pretrained models, we estimate latent state and the associated patterns of neural activity. As the patterns of neural activity become more similar to those patterns observed in the training data, we find changes in behavior and task performance that are consistent with the observations from the original, laboratory-based paradigms.Significance.These results lend ecological validity to the original, highly controlled, experimental designs and provide a methodology for understanding the relationship between neural activity and behavior during complex tasks.

Matching different-structured advertising pictorial metaphors with verbalization forms: incongruity-based evoked response potentials evidence

Many studies emphasize the need of verbally representing pictorial metaphors, but few have empirically investigated whether and how the particular verbalization form match different types of pictorial metaphors. Using evoked response potentials (ERP), a 3 (pictorial structure: fusion, juxtaposition, literal image) × 2 [verbalization form: A(is) B, A(is like) B] within-group experiment was conducted among 36 participants. ERPs were time-locked to the onset of the verb [/(is/is like)] of the metaphor sentence that follows a pictorial metaphor to detect the verbo-pictorial incongruity in metaphor comprehension. The incongruity-based ERP analysis showed that pictorial metaphors, when verbalized in two forms, all induced frontal N1 effect, regardless of pictorial structures, only with a larger N1 amplitude for literal images in "A(is) B." A central stronger P2 was observed in "A(is like) B" for three structures. Despite a general elicitation of posterior P3 in all conditions, a larger P3 was found for juxtapositions verbalized in "A(is like) B" and for literal images verbalized in "A(is) B." There was no significant difference between two verbalization forms for fusion-structured pictorial metaphors. These findings suggest: (1) verbo-pictorial metaphors could induce incongruity-based attention; (2) higher verbo-pictorial semantic congruity and relatedness, indexed by stronger P2 and P3, confirmed "A(is like) B" to be the more effective verbalization form in representing pictorial metaphors, specifically for juxtaposition-structured pictorial metaphors; (3) for non-metaphor advertising pictures, verbal metaphor showed an interference effect. The study not only reveals the neuro-cognitive mechanism of processing verbo-pictorial metaphors, but also offers neural reference for the design of effective multi-modal metaphor by finding an optimal match between PMs and verbalization forms.

The P3 indexes the distance between perceived and target image

Visual recognition requires inferring the similarity between a perceived object and a mental target. However, a measure of similarity is difficult to determine when it comes to complex stimuli such as faces. Indeed, people may notice someone "looks like" a familiar face, but find it hard to describe on the basis of what features such a comparison is based. Previous work shows that the number of similar visual elements between a face pictogram and a memorized target correlates with the P300 amplitude in the visual evoked potential. Here, we redefine similarity as the distance inferred from a latent space learned using a state-of-the-art generative adversarial neural network (GAN). A rapid serial visual presentation experiment was conducted with oddball images generated at varying distances from the target to determine how P300 amplitude related to GAN-derived distances. The results showed that distance-to-target was monotonically related to the P300, showing perceptual identification was associated with smooth, drifting image similarity. Furthermore, regression modeling indicated that while the P3a and P3b sub-components had distinct responses in location, time, and amplitude, they were similarly related to target distance. The work demonstrates that the P300 indexes the distance between perceived and target image in smooth, natural, and complex visual stimuli and shows that GANs present a novel modeling methodology for studying the relationships between stimuli, perception, and recognition.

Rapid Interactions of Widespread Brain Networks Characterize Semantic Cognition

Language comprehension requires the rapid retrieval and integration of contextually appropriate concepts ("semantic cognition"). Current neurobiological models of semantic cognition are limited by the spatial and temporal restrictions of single-modality neuroimaging and lesion approaches. This is a major impediment given the rapid sequence of processing steps that have to be coordinated to accurately comprehend language. Through the use of fused functional magnetic resonance imaging and electroencephalography analysis in humans (n = 26 adults; 15 females), we elucidate a temporally and spatially specific neurobiological model for real-time semantic cognition. We find that semantic cognition in the context of language comprehension is supported by trade-offs between widespread neural networks over the course of milliseconds. Incorporation of spatial and temporal characteristics, as well as behavioral measures, provide convergent evidence for the following progression: a hippocampal/anterior temporal phonological semantic retrieval network (peaking at ∼300 ms after the sentence final word); a frontotemporal thematic semantic network (∼400 ms); a hippocampal memory update network (∼500 ms); an inferior frontal semantic syntactic reappraisal network (∼600 ms); and nodes of the default mode network associated with conceptual coherence (∼750 ms). Additionally, in typical adults, mediatory relationships among these networks are significantly predictive of language comprehension ability. These findings provide a conceptual and methodological framework for the examination of speech and language disorders, with additional implications for the characterization of cognitive processes and clinical populations in other cognitive domains.SIGNIFICANCE STATEMENT The present study identifies a real-time neurobiological model of the meaning processes required during language comprehension (i.e., "semantic cognition"). Using a novel application of fused magnetic resonance imaging and electroencephalography in humans, we found that semantic cognition during language comprehension is supported by a rapid progression of widespread neural networks related to meaning, meaning integration, memory, reappraisal, and conceptual cohesion. Relationships among these systems were predictive of individuals' language comprehension efficiency. Our findings are the first to use fused neuroimaging analysis to elucidate language processes. In so doing, this study provides a new conceptual and methodological framework in which to characterize language processes and guide the treatment of speech and language deficits/disorders.

Ratings or Sales? The Neural and Psychological Processes of Online Experience Product Purchase: Evidence from a Sample of Chinese University Students

Extrinsic cues are ubiquitous in daily commodity consumption scenarios, not to mention online consumption scenarios. Among the many online cues, monthly sales and product ratings are two of the most representative. Some scholars have researched the impact of these cues on consumer decision making, but only search products have been investigated. Based on previous research, this article expanded the types of products to experience products and further explored consumer purchase behaviours and the underlying purchase processes influenced by these two extrinsic cues with the assistance of a neuroscience tool, event-related potentials (ERPs). The behavioural results indicated that the subjects decided mainly based on ratings, while the effect of sales was continuously inhibited. The ERP results further suggested that consumers recognised low ratings and low sales as more negative stimuli than high ratings and high sales, as larger P2 amplitudes were observed. Following the early processing of these cues, low ratings were considered unacceptable and evoked more significant emotional conflicts than high ratings, which was reflected by larger N400 amplitudes. Moreover, in the late stage, high ratings, which activated evaluation categorisation and produced more significant emotional arousal than low-rating conditions, guided the formation of purchase intention and reflected greater LPP amplitudes. Theoretical and managerial implications were discussed.

Does Eye Gaze Uniquely Trigger Spatial Orienting to Socially Relevant Information? A Behavioral and ERP Study

Using behavioral and event-related potential (ERP) measures, the present study examined whether eye gaze triggers a unique form of attentional orienting toward threat-relevant targets. A threatening or neutral target was presented after a non-predictive gaze or an arrow cue. In Experiment 1, reaction times indicated that eye gaze and arrow cues triggered different attention orienting towards threatening targets, which was confirmed by target-elicited P3b latency in Experiment 2. Specifically, for targets preceded by arrow and gaze cues, P3b peak latency was shorter for neutral targets than threatening targets. However, the latency differences were significantly smaller for gaze cues than for arrow cues. Moreover, target-elicited N2 amplitude indicated a significantly stronger cue validity effect of eye gaze than that of arrows. These findings suggest that eye gaze uniquely triggers spatial attention orienting to socially threatening information.

The influence of pride emotion on executive function: Evidence from ERP

BACKGROUND

The current study examined the influence of positive "basic" emotions on executive function; there is limited evidence about the influence of positive "self-conscious"emotions, such as pride, on executive functions processes.

METHODS

Pride is a status-related self-conscious emotion and the present research explored the influence of pride on the subcomponents of executive function, using three experiments that adopted the digit size-parity switching, N-back, and dual choice oddball paradigms.

RESULTS

The behavioral results suggested that cognitive load and behavior inhibition effects in the pride emotion were significantly higher than the neutral emotion. The ERP results showed that the pride emotion elicited smaller P3 difference wave for the switching task and dual choice oddball task. In the N-back task, the pride emotion elicited larger N1 amplitude and smaller P2 difference wave compared to the neutral emotion. A comparison among results from the three experiments indicated that pride emotion restrains all subcomponents of executive function, though with different manifestations of the impact.

CONCLUSION

Experiencing positive emotions is typically viewed as desirable and adaptive in educational settings; however, pride as a unique positive emotion may damage people's cognitive performance, indicating that we need to be cautious when performing cognitive operations in a pride mood.

How does the use of simultaneous contrast illusion on product-background color combination nudge consumer behavior? A behavioral and event-related potential study

Color, as one of the most critical visual factors influencing consumer decisions, has been widely used in e-commerce marketing. However, the effects of product-background saturation combination on consumers’ willingness to purchase products with different heaviness attributes (e.g., heaviness-positive products or heaviness-negative products) have not been conclusively determined. The current study demonstrated the effects of product-background saturation combination on product heaviness perception and its downstream consequences. Based on behavioral method, study 1 showed that a patch of color placed in a pale background (the saturation of the background is lower than the saturation of the color patch) was perceived as visually heavier than that in a colorful background (the saturation of the background is higher than the saturation of the color patch). Study 2 applied event-related potentials (ERPs) method to explore the underlying neural mechanisms of how the interactions between the presentation modes and the product types affect consumer decisions. Behaviorally, compared to the colorful background, the pale background would lead to a higher purchase rate for the heaviness-positive products, whereas the opposite results were found for the heaviness-negative products. Furthermore, for both the heaviness-positive and heaviness-negative products, a shorter reaction time would be observed in the pale background condition than in the colorful background condition. Neurophysiologically, the pale background would result in smaller N2 component and larger P3 component compared to the colorful background for the heaviness-positive products, while the reverse held for the heaviness-negative products. Smaller N2 component implies decreased perceptual conflicts and larger P3 component implies increased decision confidence, suggesting that e-retailers should present heaviness-positive products with pale backgrounds and heaviness-negative products with colorful backgrounds.

Different Educational Interventions on Individual Cognition of Garbage Classification Based on EEG Monitoring

Improvement in an individuals’ cognition is the key to promote garbage classification. This study takes university students as the research subjects, through three educational interventions, including the self-learning, heuristic learning, and interactive learning ways, to seek the most effective intervention based upon event-related potentials (ERPs) that is beneficial to enhance cognition of garbage classification. The results show that the experimental subjects induced P300 and LPP components, representing attentional changes and cognitive conflicts in classification judgments. There are differences in the amplitudes and peak latency of the two components corresponding to different interventions, indicating that the three educational interventions are able to improve the individual’s cognition level of garbage classification within a certain period of time. The interactive-learning intervention triggers the largest amplitudes of P300 and LPP, as well as the smallest peak latency, indicating its effect is the best. Such results provide insight into the design for an appropriate strategy in garbage classification education. The study also shows that an EEG signal can be used as the endogenous neural indicator to measure the performance of garbage classification under different educational interventions.

Is Distant Extension Always Upset? Neural Evidence of Empathy and Brand Association Affect Distant Extension Evaluation

Distant brand extension as an essential strategy of obtaining benefits was highly focused on the normal marketing practice and academic research. In the current study, we aim to recognize that how individuals with different levels of empathy respond to distant extensions under corporate social responsibility (CSR) and corporate competence (CC) associations to explore the corresponding neural mechanisms using event-related potentials (ERPs). We divided subjects into two groups involving a high empathy (HE) group and a low empathy (LE) group according to an empathy measure questionnaire. The subjects first faced a brand name following the CSR or CC association descriptions, and then, they were asked to evaluate the new product of brand by a five-point scale. Current results revealed that the participants of the HE group were more apt to accept the distant extension products than those of the LE group. Additionally, in the HE group, products from a brand with CSR associations were more acceptable than CC associations. Moreover, a larger N2 amplitude was elicited in the LE group than in the HE group. For the LE group, an augment N2 was found under CSR than CC associations, reflecting that LE consumers might perceive conflict when evaluating distant extensions and allocate more cognitive resources to deal with CSR information. At the later stage, the HE group showed a greater P3 than the LE group. For the HE group, an increased P3 was elicited under CSR than CC associations, suggesting that empathic individuals might show motivational salience and helping willingness toward distant extension products, especially under the CSR scenario. These results provide potential electrophysiological evidence for the positive impact of brand associations on the evaluation of distant brand extension in the case of subdividing different empathic individuals.

Banknote authenticity is signalled by rapid neural responses

Authenticating valuable objects is widely assumed to involve protracted scrutiny for detection of reproduction flaws. Yet, accurate authentication of banknotes is possible within one second of viewing, suggesting that rapid neural processes may underpin counterfeit detection. To investigate, we measured event-related brain potentials (ERPs) in response to briefly viewed genuine or forensically recovered counterfeit banknotes presented in a visual oddball counterfeit detection task. Three ERP components, P1, P3, and extended P3, were assessed for each combination of banknote type (genuine, counterfeit) and overt response (“real”, “fake”). P1 amplitude was greater for oddballs, demonstrating that the initial feedforward sweep of visual processing yields the essential information for differentiating genuine from counterfeit. A similar oddball effect was found for P3. The magnitude of this P3 effect was positively correlated with behavioural counterfeit sensitivity, although the corresponding correlation for P1 was not. For the extended P3, amplitude was greatest for correctly detected counterfeits and similarly small for missed counterfeits, incorrectly and correctly categorised genuine banknotes. These results show that authentication of complex stimuli involves a cascade of neural processes that unfolds in under a second, beginning with a very rapid sensory analysis, followed by a later decision stage requiring higher level processing.

Importance of the Features of Event-Related Potentials Used for a Machine Learning-Based Model Applied to Single-Trial Data during Oddball Task

In this study, a method for assessing the human state and brain-machine interface (BMI) has been developed using event-related potentials (ERPs). Most of these algorithms are classified based on the ERP characteristics. To observe the characteristics of ERPs, an averaging method using electroencephalography (EEG) signals cut out by time-locking to the event for each condition is required. To date, several classification methods using only single-trial EEG signals have been studied. In some cases, the machine learning models were used for the classifications; however, the relationship between the constructed model and the characteristics of ERPs remains unclear. In this study, the LightGBM model was constructed for each individual to classify a single-trial waveform and visualize the relationship between these features and the characteristics of ERPs. The features used in the model were the average values and standard deviation of the EEG amplitude with a time width of 10 ms. The best area under the curve (AUC) score was 0.92, but, in some cases, the AUC scores were low. Large individual differences in AUC scores were observed. In each case, on checking the importance of the features, high importance was shown at the 10-ms time width section, where a large difference was observed in ERP waveforms between the target and the non-target. Since the model constructed in this study was found to reflect the characteristics of ERP, as the next step, we would like to try to improve the discrimination performance by using stimuli that the participants can concentrate on with interest.

The mechanism of filler items in the response time concealed information test

The response time concealed information test (RT-CIT) can reveal that a person recognizes a relevant (probe) item among other, irrelevant items, based on slower responding to the probe compared to the irrelevant items. Therefore, if this person is concealing the knowledge about the relevance of this item (e.g., recognizing it as a murder weapon), this deception can be unveiled. Adding familiarity-related filler items to the task has been shown to substantially increase the validity of the method, but assumptions for this effect have never been tested before. In the present series of three experiments (N = 511), we tested several factors, most of which were found to indeed influence the enhancing effects of fillers. First, larger enhancement is achieved when a smaller proportion of fillers shares the response key with the target. Second, familiarity context does play a role in the enhancement, and the target sharing its response key with the familiarity-referring fillers leads to larger enhancement. Third, mere symbolic fillers (such as simple arrow-like characters) also lead to enhancement, but filler words without task-relevant meaning are not effective. Fourth, small visual differences (lettercase or underlining) between fillers and the rest of the items have no significant influence. All this provides justification for the original structure of the fillers and also demonstrates that the enhancement is highly generalizable: Fillers have a potential to improve the RT-CIT regardless of deception scenario, item types, or the examinee's language comprehension.

Timing of readiness potentials reflect a decision-making process in the human brain

Decision-making in two-alternative forced choice tasks has several underlying components including stimulus encoding, perceptual categorization, response selection, and response execution. Sequential sampling models of decision-making are based on an evidence accumulation process to a decision boundary. Animal and human studies have focused on perceptual categorization and provide evidence linking brain signals in parietal cortex to the evidence accumulation process. In this exploratory study, we use a task where the dominant contribution to response time is response selection and model the response time data with the drift-diffusion model. EEG measurement during the task show that the Readiness Potential (RP) recorded over motor areas has timing consistent with the evidence accumulation process. The duration of the RP predicts decision-making time, the duration of evidence accumulation, suggesting that the RP partly reflects an evidence accumulation process for response selection in the motor system. Thus, evidence accumulation may be a neural implementation of decision-making processes in both perceptual and motor systems. The contributions of perceptual categorization and response selection to evidence accumulation processes in decision-making tasks can be potentially evaluated by examining the timing of perceptual and motor EEG signals.

Enhancement of Processing Capabilities of Hippocampus Lobe: A P300 Based Event Related Potential Study

Background and Objectives

The influence of music training on different areas of the brain has been extensively researched, but the underlying neurobehavioral mechanisms remain unknown. In the present study, the effects of training for more than three years in Carnatic music (an Indian form of music) on the discrimination ability of different areas of the brain were tested using P300 analysis at three electrode placement sites.

Subjects and Methods

A total of 27 individuals, including 13 singers aged 16-30 years (mean±standard deviation, 23±3.2 years) and 14 non-singers aged 16-30 years (mean age, 24±2.9 years), participated in this study. The singers had 3-5 years of formal training experience in Carnatic music. Cortical activities in areas corresponding to attention, discrimination, and memory were tested using P300 analysis, and the tests were performed using the Intelligent Hearing System.

Results

The mean P300 amplitude of the singers at the Fz electrode placement site (5.64±1.81) was significantly higher than that of the non-singers (3.85±1.60; t(25)=3.3, p<0.05). The amplitude at the Cz electrode placement site in singers (5.90±2.18) was significantly higher than that in non-singers (3.46±1.40; t(25)=3.3, p<0.05). The amplitude at the Pz electrode placement site in singers (4.94±1.89) was significantly higher than that in non-singers (3.57±1.50; t(25)=3.3, p<0.05). Among singers, the mean P300 amplitude was significantly higher in the Cz site than the other placement sites, and among non-singers, the mean P300 amplitude was significantly higher in the Fz site than the other placement sites, i.e., music training facilitated enhancement of the P300 amplitude at the Cz site.

Conclusions

The findings of this study suggest that more than three years of training in Carnatic singing can enhance neural coding to discriminate subtle differences, leading to enhanced discrimination abilities of the brain, mainly in the generation site corresponding to Cz electrode placement.

Decreased P2 Waveform Reflects Impaired Brain Executive Function Induced by 12 h of Low Homeostatic Sleep Pressure: Evidence From an Event-Related Potential Study

Homeostatic sleep pressure can cause cognitive impairment, in which executive function is the most affected. Previous studies have mainly focused on high homeostatic sleep pressure (long-term sleep deprivation); thus, there is still little related neuro-psycho-physiological evidence based on low homeostatic sleep pressure (12 h of continuous wakefulness) that affects executive function. This study aimed to investigate the impact of lower homeostatic sleep pressure on executive function. Our study included 14 healthy young male participants tested using the Go/NoGo task in normal resting wakefulness (10:00 am) and after low homeostatic sleep pressure (10:00 pm). Behavioral data (response time and accuracy) were collected, and electroencephalogram (EEG) data were recorded simultaneously, using repeated measures analysis of variance for data analysis. Compared with resting wakefulness, the participants' response time to the Go stimulus was shortened after low homeostatic sleep pressure, and the correct response rate was reduced. Furthermore, the peak amplitude of Go-P2 decreased significantly, and the peak latency did not change significantly. For NoGo stimulation, the peak amplitude of NoGo-P2 decreased significantly (p < 0.05), and the peak latency was significantly extended (p < 0.05). Thus, the P2 wave is likely related to the attention and visual processing and reflects the early judgment of the perceptual process. Therefore, the peak amplitude of Go-P2 and NoGo-P2 decreased, whereas the peak latency of NoGo-P2 increased, indicating that executive function is impaired after low homeostatic sleep pressure. This study has shown that the P2 wave is a sensitive indicator that reflects the effects of low homeostatic sleep pressure on executive function, and that it is also an important window to observe the effect of homeostatic sleep pressure and circadian rhythm on cognitive function.

Effect of inhibition indexed by auditory P300 on transmission of visual sensory information

Early electroencephalographic studies that focused on finding brain correlates of psychic events led to the discovery of the P300. Since then, the P300 has become the focus of many basic and clinical neuroscience studies. However, despite its wide applications, the underlying function of the P300 is not yet clearly understood. One line of research among the many studies that have attempted to elucidate the underlying subroutine of the P300 in the brain has suggested that the physiological function of the P300 is related to inhibition. While some intracranial, behavioral, and event-related potential studies have provided support for this theory, little is known about the inhibitory mechanism. In this study, using alpha event-related desynchronization (ERD) and effective connectivity, based on the causal (one-way directed) relationship between alpha ERD and P300 sources, we demonstrated that P300's associated inhibition is implemented at a higher information processing stage in a localized brain region. We discuss how inhibition as the primary function of the P300 is not inconsistent with 'resource allocation' and 'working memory updating' theories about its cognitive function. In light of our findings regarding the scope and information processing stage of inhibition of the P300, we reconcile the inhibitory account of the P300 with working memory updating theory. Finally, based on the compensatory behavior of alpha ERD at the time of suppression of the P300, we propose two distinct yet complementary working memory mechanisms (inhibition and desynchronizing excitation) that render target perception possible.

Natural Categorization: Electrophysiological Responses to Viewing Natural Versus Built Environments

Environments are unique in terms of structural composition and evoked human experience. Previous studies suggest that natural compared to built environments may increase positive emotions. Humans in natural environments also demonstrate greater performance on attention-based tasks. Few studies have investigated cortical mechanisms underlying these phenomena or probed these differences from a neural perspective. Using a temporally sensitive electrophysiological approach, we employ an event-related, implicit passive viewing task to demonstrate that in humans, a greater late positive potential (LPP) occurs with exposure to built than natural environments, resulting in a faster return of activation to pre-stimulus baseline levels when viewing natural environments. Our research thus provides new evidence suggesting natural environments are perceived differently from built environments, converging with previous behavioral findings and theoretical assumptions from environmental psychology.

Language can shape the perception of oriented objects

Seeing an object is a natural source for learning about the object's configuration. We show that language can also shape our knowledge about visual objects. We investigated sign language that enables deaf individuals to communicate through hand movements with as much expressive power as any other natural language. A few signs represent objects in a specific orientation. Sign-language users (signers) recognized visual objects faster when oriented as in the sign, and this match in orientation elicited specific brain responses in signers, as measured by event-related potentials (ERPs). Further analyses suggested that signers' responsiveness to object orientation derived from changes in the visual object representations induced by the signs. Our results also show that language facilitates discrimination between objects of the same kind (e.g., different cars), an effect never reported before with spoken languages. By focusing on sign language we could better characterize the impact of language (a uniquely human ability) on object visual processing.

Distinguishing the Neural Correlates of Perceptual Awareness and Postperceptual Processing

To identify the neural correlates of perceptual awareness, researchers often compare the differences in neural activation between conditions in which an observer is or is not aware of a stimulus. While intuitive, this approach often contains a critical limitation: to link brain activity with perceptual awareness, observers traditionally report the contents of their perceptual experience. However, relying on observers' reports is problematic because it is difficult to know whether the neural responses being measured are associated with conscious perception or with postperceptual processes involved in the reporting task (e.g., working memory, decision-making). To address this issue, we combined a standard visual masking paradigm with a recently developed "no-report" paradigm in male/female human participants. In the visual masking paradigm, observers saw images of animals and objects that were visible or invisible, depending on their proximity to masks. Meanwhile, on half of the trials, observers reported the contents of their perceptual experience (i.e., report condition), while on the other half of trials they refrained from reporting about their experiences (i.e., no-report condition). We used electroencephalography to examine how visibility interacts with reporting by measuring the P3b event-related potential, one of the proposed canonical "signatures" of conscious processing. Overall, we found a robust P3b in the report condition, but no P3b whatsoever in the no-report condition. This finding suggests that the P3b itself is not a neural signature of conscious processing and highlights the importance of carefully distinguishing the neural correlates of perceptual awareness from postperceptual processing.SIGNIFICANCE STATEMENT What are the neural signatures that differentiate conscious and unconscious processing in the brain? Perhaps the most well established candidate signature is the P3b event-related potential, a late slow wave that appears when observers are aware of a stimulus, but disappears when a stimulus fails to reach awareness. Here, however, we found that the P3b does not track what observers are perceiving, but instead tracks what observers are reporting. When observers are aware of simple visual stimuli, the P3b is nowhere to be found unless observers are reporting the contents of their experience. These results challenge the well established notion of the P3b as a neural marker of awareness and highlight the need for new approaches to the neuroscience of consciousness.

Muscular effort coding in action representation in ballet dancers and controls: Electrophysiological evidence

The present electrophysiological (EEG) study investigated the neural correlates of perceiving effortful vs. effortless movements belonging to a specific repertoire (ballet). Previous evidence has shown an increased heart and respiratory rate during the observation and imagination of human actions that require a great muscular effort. In addition, TMS (transcranial magnetic stimulation) and EEG studies have evidenced a greater muscle-specific cortical excitability and an increase in late event-related potentials during the observation of effortful actions. In this investigation, fifteen professional female ballet dancers and 15 controls with no experience whatsoever with dance, gymnastics, or martial arts were recruited. They were shown 326 short videos displaying a male dancer performing standard ballet steps that could be either effortful or relatively effortless. Participants were instructed to observe each clip and imagine themselves physically executing the same movement. Importantly, they were blinded to the stimuli properties. The observation of effortful compared with effortless movements resulted in a larger P300 over frontal sites in dancers only, likely because of their visuomotor expertise with the specific steps. Moreover, an enhanced Late Positivity was identified over posterior sites in response to effortful stimuli in both groups, possibly reflecting the processing of larger quantities of visual kinematic information. The source reconstruction swLORETA performed on the Late Positivity component showed greater engagement of frontoparietal regions in dancers, while task-related frontal and occipitotemporal visual regions were more active in controls. It, therefore, appears that, in dancers, effort information was encoded in a more refined manner during action observation and in the absence of explicit instruction. Acquired motor knowledge seems to result in visuomotor resonance processes, which, in turn, underlies enhanced action representation of the observed movements.

Central auditory processing in parkinsonian disorders: A systematic review

Altered auditory processing has been increasingly recognized as a non-motor feature in parkinsonian disorders. This systematic review provides an overview of behavioral and electrophysiological literature on central auditory processing in patients with Parkinson's disease (PD), multiple system atrophy (MSA) and progressive supranuclear palsy (PSP). A systematic database search was conducted and yielded 88 studies that met the intelligibility criteria. The collected data revealed distinct impairments in a range of central auditory processes in PD, including altered deviance detection of basic auditory features, auditory brainstem processing, auditory gating and selective auditory attention. In contrast to PD, literature on central auditory processing in atypical parkinsonian disorders was relatively scarce, but provided some evidence for impaired central auditory processing in MSA and PSP. The interpretation of these findings is discussed and suggestions for further research are offered.

Left-Hemispheric Asymmetry for Object-Based Attention: an ERP Study

It has been shown that selective attention enhances the activity in visual regions associated with stimulus processing. The left hemisphere seems to have a prominent role when non-spatial attention is directed towards specific stimulus features (e.g., color, spatial frequency). The present electrophysiological study investigated the time course and neural correlates of object-based attention, under the assumption of left-hemispheric asymmetry. Twenty-nine right-handed participants were presented with 3D graphic images representing the shapes of different object categories (wooden dummies, chairs, structures of cubes) which lacked detail. They were instructed to press a button in response to a target stimulus indicated at the beginning of each run. The perception of non-target stimuli elicited a larger anterior N2 component, which was likely associated with motor inhibition. Conversely, target selection resulted in an enhanced selection negativity (SN) response lateralized over the left occipito-temporal regions, followed by a larger centro-parietal P300 response. These potentials were interpreted as indexing attentional selection and categorization processes, respectively. The standardized weighted low-resolution electromagnetic tomography (swLORETA) source reconstruction showed the engagement of a fronto-temporo-limbic network underlying object-based visual attention. Overall, the SN scalp distribution and relative neural generators hinted at a left-hemispheric advantage for non-spatial object-based visual attention.

Extended categorization of conjunction object stimuli decreases the latency of attentional feature selection and recruits orthography-linked ERPs

The role of attention in driving perceptual expertise effects is controversial. The current study addressed the effect of training on ERP components related to and independent of attentional feature selection. Participants learned to categorize cartoon animals over six training sessions (8,800 trials) after which ERPs were recorded during a target detection task performed on trained and untrained stimulus sets. The onset of the selection negativity, an ERP component indexing attentional modulation, was about 60 msec earlier for trained than untrained stimuli. Trained stimuli also elicited centro-parietal N200 and N320 components that were insensitive to attentional feature selection. The scalp distribution and timecourse of these components were better matched by studies of orthography than object expertise. Source localization using eLORETA suggested that the strongest neural sources of the selection negativity were in right ventral temporal cortex whereas the strongest sources of the N200/N320 components were in left ventral temporal cortex, again consistent with the hypothesis that training recruited orthography related areas. Overall, training altered neural processes related to attentional selection, but also affected neural processes that were independent of feature selection.

Decoding P300 Variability Using Convolutional Neural Networks

Deep convolutional neural networks (CNN) have previously been shown to be useful tools for signal decoding and analysis in a variety of complex domains, such as image processing and speech recognition. By learning from large amounts of data, the representations encoded by these deep networks are often invariant to moderate changes in the underlying feature spaces. Recently, we proposed a CNN architecture that could be applied to electroencephalogram (EEG) decoding and analysis. In this article, we train our CNN model using data from prior experiments in order to later decode the P300 evoked response from an unseen, hold-out experiment. We analyze the CNN output as a function of the underlying variability in the P300 response and demonstrate that the CNN output is sensitive to the experiment-induced changes in the neural response. We then assess the utility of our approach as a means of improving the overall signal-to-noise ratio in the EEG record. Finally, we show an example of how CNN-based decoding can be applied to the analysis of complex data.

The Sound and Face of Others: Vocal Priming Effects on Facial Emotion Processing in Posttraumatic Stress Disorder

INTRODUCTION

Facial expressions and vocal intonation are key signals in the communication of emotions. Individuals with posttraumatic stress disorder (PTSD) are known to show an impaired perception of facial emotions. So far, research on multimodal emotional stimuli or the priming effects on emotion processing has been absent in PTSD. Therefore, we conducted a study to investigate the influence of vocal priming on facial emotion processing and classification in PTSD using electroencephalography.

METHODS

Twenty-one women with PTSD compared to 28 healthy women were asked to classify emotion-morphed faces with predominantly angry, ambiguous, or predominantly happy expressions primed by either an angry or a happy voice. Responses and reaction times as well as the N170, a component reflecting configural face processing, were analyzed.

RESULTS

Patients with PTSD were slower in classifying emotional faces that were primed by either an angry or happy voice compared to the healthy controls (HCs; η2 = 0.14). Additionally, patients with PTSD were faster in classifying facial expressions after angry compared to happy vocal primes (η2 = 0.14). HCs did not show this effect. Correlation analyses revealed positive associations between emotion (dys-)regulation and reaction times in patients with PTSD but not in HCs (r = 0.64-0.76). Furthermore, patients with PTSD showed greater N170 amplitudes for predominantly angry and ambiguous faces than HCs (η2 = 0.07).

CONCLUSION

Data suggest that patients with PTSD experience more difficulties when processing complex social stimuli than HCs. The altered processing of complex social-emotional signals could amplify PTSD symptoms, thus qualifying as an explicit therapy target.

Analyzing P300 Distractors for Target Reconstruction

P300-based brain-computer interfaces (BCIs) are often trained per-user and per-application space. Training such models requires ground truth knowledge of target and nontarget stimulus categories during model training, which imparts bias into the model. Additionally, not all non-targets are created equal; some may contain visual features that resemble targets or may otherwise be visually salient. Current research has indicated that non-target distractors may elicit attenuated P300 responses based on the perceptual similarity of these distractors to the target category. To minimize this bias, and enable a more nuanced analysis, we use a generalized BCI approach that is fit to neither user nor task. We do not seek to improve the overall accuracy of the BCI with our generalized approach; we instead demonstrate the utility of our approach for identifying targetrelated image features. When combined with other intelligent agents, such as computer vision systems, the performance of the generalized model equals that of the user-specific models, without any user specific data.

Remnants and changes in facial emotion processing in women with remitted borderline personality disorder: an EEG study

According to longitudinal studies, most individuals with borderline personality disorder (BPD) achieve remission. Since BPD is characterized by disturbed emotion recognition, this study investigated behavioral and electrophysiological correlates of facial emotion classification and processing in remitted BPD. 32 women with remitted BPD (rBPD), 32 women with current BPD (cBPD), and 28 healthy women (HC) participated in an emotion classification paradigm comprising blends of angry and happy faces while behavioral and electroencephalographic (event-related potentials) data were recorded. rBPD demonstrated a convergence in behavior towards HC in terms of responses and reaction times. They evaluated maximally ambiguous faces more positively and exhibited faster reaction times when classifying predominantly happy faces compared to cBPD. Group × facial emotion interaction effects were found in early electrophysiological processes with post hoc tests indicating differences between rBPD and cBPD but not between rBPD and HC. However, BPD-like impairments were still found in rBPD in later processing (P300). Our results suggest a reduction in negativity bias in rBPD on the behavioral level and a normalization of earlier stages of facial processing on the neural level, while alterations in later, more cognitive processing do not remit. Early processing may be more state-like, while later impairments may be more trait-like. Further research may need to focus on these stable components.

Characteristics of Human Brain Activity during the Evaluation of Service-to-Service Brand Extension

Brand extension is a marketing strategy to apply the previously established brand name into new goods or service. A number of studies have reported the characteristics of human event-related potentials (ERPs) in response to the evaluation of goods-to-goods brand extension. In contrast, human brain responses to the evaluation of service extension are relatively unexplored. The aim of this study was investigating cognitive processes underlying the evaluation of service-to-service brand extension with electroencephalography (EEG). A total of 56 text stimuli composed of service brand name (S1) followed by extended service name (S2) were presented to participants. The EEG of participants was recorded while participants were asked to evaluate whether a given brand extension was acceptable or not. The behavioral results revealed that participants could evaluate brand extension though they had little knowledge about the extended services, indicating the role of brand in the evaluation of the services. Additionally, we developed a method of grouping brand extension stimuli according to the fit levels obtained from behavioral responses, instead of grouping of stimuli a priori. The ERP analysis identified three components during the evaluation of brand extension: N2, P300, and N400. No difference in the N2 amplitude was found among the different levels of a fit between S1 and S2. The P300 amplitude for the low level of fit was greater than those for higher levels (p < 0.05). The N400 amplitude was more negative for the mid- and high-level fits than the low level. The ERP results of P300 and N400 indicate that the early stage of brain extension evaluation might first detect low-fit brand extension as an improbable target followed by the late stage of the integration of S2 into S1. Along with previous findings, our results demonstrate different cognitive evaluation of service-to-service brand extension from goods-to-goods.

The effect of category learning on attentional modulation of visual cortex

Learning about visual object categories causes changes in the way we perceive those objects. One likely mechanism by which this occurs is the application of attention to potentially relevant objects. Here we test the hypothesis that category membership influences the allocation of attention, allowing attention to be applied not only to object features, but to entire categories. Participants briefly learned to categorize a set of novel cartoon animals after which EEG was recorded while participants distinguished between a target and non-target category. A second identical EEG session was conducted after two sessions of categorization practice. The category structure and task design allowed parametric manipulation of number of target features while holding feature frequency and category membership constant. We found no evidence that category membership influenced attentional selection: a postero-lateral negative component, labeled the selection negativity/N250, increased over time and was sensitive to number of target features, not target categories. In contrast, the right hemisphere N170 was not sensitive to target features. The P300 appeared sensitive to category in the first session, but showed a graded sensitivity to number of target features in the second session, possibly suggesting a transition from rule-based to similarity based categorization.

Changing similarity: Stable and flexible modulations of psychological dimensions

Successfully categorizing objects requires discriminating between relevant and irrelevant dimensions (e.g., shape, color). Categorization can lead to changes in the visual system that stretch psychological space, making relevant dimensions more distinct and irrelevant dimensions more similar. These changes are known as dimensional modulation (DM) and they can be both stable and flexible in nature. The current study examined the interaction between stable DM and flexible DM, as well as the time course of relative changes in similarity. Using a two-dimensional space of cars, participants learned to categorize the space and then completed a target identification task during EEG recording. We found that attention, operationally defined as the selection negativity, was sensitive to category-relevance and appeared to selectively enhance previously irrelevant differences in the service of a target detection task. In contrast, we found that late decisional stages, operationally defined as the P3 b, were less sensitive to relevance and instead more sensitive to the number of morphsteps that separated targets from non-targets. Thus, it appears that relative similarity between targets and non-targets dynamically changed over the time course of individual decisions. Similarity between exemplars was greater along the irrelevant than the relevant dimension early on in the time course but a compensatory allocation of attention led to similarity being optimized among all dimensions for later stages. This finding is important because it 1) provides a new source of converging evidence for stable DM and 2) links a neural measure of attentional modulation with facilitation of an unpracticed, but task-relevant perceptual dimension.

Event-related potential measures of executive functioning from preschool to adolescence

Executive functions are a collection of cognitive abilities necessary for behavioural control and regulation, and are important for school success. Executive deficits are common across acquired and developmental disorders in childhood and beyond. This review aims to summarize how studies using event-related potential (ERP) can provide insight into mechanisms underpinning how executive functions develop in children from preschool to adolescence. We specifically focus on ERP components that are considered to be well-established markers of executive functioning, including the ability to resist distraction (inhibition, N200), hold scenes in mind (visuospatial working memory, contralateral delay activity), attend to specific stimuli (information processing, P300), follow rules (response monitoring, error-related negativity [ERN], and error-related positivity [Pe]), and adjust to feedback (outcome monitoring, feedback-related negativity). All of these components show developmental changes from preschool to adolescence, in line with behavioural and neuroimaging findings. These ERP markers also show altered developmental trajectories in the context of atypical executive functions. As an example, deficits in executive function are prominently implicated in attention-deficit-hyperactivity disorder. Therefore, this review highlights ERP studies that have investigated the above ERP components in this population. Overall, ERPs provide a useful marker for the development and dysfunction of executive skills, and provide insight into their neurophysiological basis.

Logo Effects on Brand Extension Evaluations from the Electrophysiological Perspective

Brand extension typically has two strategies: brand name extension (BN) and brand logo extension (BL). The current study explored which strategy (BN or BL) better enhanced the success of dissimilar brand extension and product promotion in enterprises. Event-related potentials (ERPs) were used to investigate electrophysiological processes when subjects evaluated their acceptance of the brand extension using a combined picture of S1 and S2. S1 was a famous brand presented by two identity signs (brand name and brand logo). S2 was a picture of an extension product that belonged to a dissimilar product category than S1. The behavior data showed that BL was more acceptable than BN in the dissimilar brand extension. The neurophysiology process was reflected by a less negative N2 component and a larger P300 component in the BL than in the BN. We suggested that N2 reflected a whole conflict between the brand-product combination and the long-term memory and that P300 could be regarded as the reflection of the categorization process in the working memory.

The Neural Correlates Underlying Belief Reasoning for Self and for Others: Evidence from ERPs

Belief reasoning is typical mental state reasoning in theory of mind (ToM). Although previous studies have explored the neural bases of belief reasoning, the neural correlates of belief reasoning for self and for others are rarely addressed. The decoupling mechanism of distinguishing the mental state of others from one’s own is essential for ToM processing. To address the electrophysiological bases underlying the decoupling mechanism, the present event-related potential study compared the time course of neural activities associated with belief reasoning for self and for others when the belief belonging to self was consistent or inconsistent with others. Results showed that during a 450–600 ms period, belief reasoning for self elicited a larger late positive component (LPC) than for others when beliefs were inconsistent with each other. The LPC divergence is assumed to reflect the categorization of agencies in ToM processes.

Is anterior N2 enhancement a reliable electrophysiological index of concealed information?

Concealed information tests (CITs) are used to determine whether an individual possesses information about an item of interest. Event-related potential (ERP) measures in CITs have focused almost exclusively on the P3b component, showing that this component is larger when lying about the item of interest (probe) than telling the truth about control items (irrelevants). Recent studies have begun to examine other ERP components, such as the anterior N2, with mixed results. A seminal CIT study found that visual probes elicit a larger anterior N2 than irrelevants (Gamer and Berti, 2010) and suggested that this component indexes cognitive control processes engaged when lying about probes. However, this study did not control for potential intrinsic differences among the stimuli: the same probe and irrelevants were used for all participants, and there was no control condition composed of uninformed participants. Here, first we show that the N2 effect found in the study by Gamer and Berti (2010) was in large part due to stimulus differences, as the effect observed in a concealed information condition was comparable to that found in two matched control conditions without any concealed information (Experiments 1 and 2). Next, we addressed the issue of the generality of the N2 findings by counterbalancing a new set of stimuli across participants and by using a control condition with uninformed participants (Experiment 3). Results show that the probe did not elicit a larger anterior N2 than the irrelevants under these controlled conditions. These findings suggest that caution should be taken in using the N2 as an index of concealed information in CITs. Furthermore, they are a reminder that results of CIT studies (not only with ERPs) performed without stimulus counterbalancing and suitable control conditions may be confounded by differential intrinsic properties of the stimuli employed.

Online processing of tone and intonation in Mandarin: Evidence from ERPs

Event-related potentials (ERPs) were used to investigate the online processing of tone and intonation in Mandarin at the attentive stage. We examined the behavioral and electrophysiological responses of native Mandarin listeners to Mandarin sentences, which contrast in final tones (rising Tone2 or falling Tone4) and intonations (Question or Statement). A clear P300 effect was observed for question-statement contrast in sentences ending with Tone4, but no ERP effect was found for question-statement contrast in sentences ending with Tone2. Our results provide ERP evidence for the interaction of tone and intonation in Mandarin, confirming the findings with behavioral metalinguistic data that native Mandarin listeners can distinguish between question intonation and statement intonation when the intonation is associated with a final Tone4, but fail to do so when the intonation is associated with a final Tone2. Our study extended the understanding of online processing of tone and intonation (1) from the pre-attentive stage to the attentive stage and (2) within a larger domain (i.e. multi-word utterances) than a single word utterance.

The impact of perceived quality on online buying decisions: an event-related potentials perspective

Consumer neuroscience can provide useful insights into the neural foundations of consumer decisions, such as perceived quality. One of the applications is to guide attribute configuration of products to fit consumers' expectations on the basis of individual preferences. In this study, we required 20 participants to decide whether to buy the product provided in the stimuli and to respond as soon as possible. According to their reports of expectations after the experiment, we subdivided the stimuli into two conditions. Condition 1 contained the stimuli that fit individual preferences, whereas Condition 2 contained the other stimuli. An essential component of event-related potentials (ERPs), the P300, was elicited in the two conditions and distributed over almost all parietal and occipital regions. Products in Condition 1 induced a higher P300 amplitude than those in Condition 2. The results show that evaluating product attributes is a cognitive process that modulates attention in the aforementioned regions. When participants evaluate the alternatives, categorical processing occurred on the basis of similarity judgment. The situation in Condition 1 produced a similarity overlap between the product and the expectation and resulted in a higher P300. Otherwise, there was no overlap, leading to a smaller P300. Hence, the P300 may be a useful neural endogenous indicator for measuring consumers' evaluations of products in marketing research.

Gamma band activity and the P3 reflect post-perceptual processes, not visual awareness

A primary goal in cognitive neuroscience is to identify neural correlates of conscious perception (NCC). By contrasting conditions in which subjects are aware versus unaware of identical visual stimuli, a number of candidate NCCs have emerged; among them are induced gamma band activity in the EEG and the P3 event-related potential. In most previous studies, however, the critical stimuli were always directly relevant to the subjects' task, such that aware versus unaware contrasts may well have included differences in post-perceptual processing in addition to differences in conscious perception per se. Here, in a series of EEG experiments, visual awareness and task relevance were manipulated independently. Induced gamma activity and the P3 were absent for task-irrelevant stimuli regardless of whether subjects were aware of such stimuli. For task-relevant stimuli, gamma and the P3 were robust and dissociable, indicating that each reflects distinct post-perceptual processes necessary for carrying-out the task but not for consciously perceiving the stimuli. Overall, this pattern of results challenges a number of previous proposals linking gamma band activity and the P3 to conscious perception.

Cortical potentials in an auditory oddball task reflect individual differences in working memory capacity

This study determined whether auditory cortical responses associated with mechanisms of attention vary with individual differences in working memory capacity (WMC) and perceptual load. The operation span test defined subjects with low versus high WMC, who then discriminated target/nontarget tones while EEG was recorded. Infrequent white noise distracters were presented at midline or ±90° locations, and perceptual load was manipulated by varying nontarget frequency. Amplitude of the N100 to distracters was negatively correlated with WMC. Relative to targets, only high WMC subjects showed attenuated N100 amplitudes to nontargets. In the higher WMC group, increased perceptual load was associated with decreased P3a amplitudes to distracters and longer-lasting negative slow wave to nontargets. Results show that auditory cortical processing is associated with multiple facets of attention related to WMC and possibly higher-level cognition.

Impact of stimulus similarity between the probe and the irrelevant items during a card-playing deception detection task: the "irrelevants" are not irrelevant

Event-related brain potential paradigms for the detection of concealed information commonly involve presenting probes embedded within a series of irrelevant items. This study investigated the impact of similarity of the irrelevant items with the probe. For the task, a card was shown followed by the sequential presentation of six "test" cards, one of which was the same as the initial card (the probe) along with five "irrelevant" cards that varied in terms of similarity with the probe. Participants either identified or denied recognition of the probe. The results show that P300 amplitude is modulated by stimulus similarity and highlight the importance of the irrelevant items on deception detection rates.