Applying deep learning to single-trial EEG data provides evidence for complementary theories on action control

Effect of natural and synthetic noise data augmentation on physical action classification by brain-computer interface and deep learning

Analysis of electroencephalography (EEG) signals gathered by brain-computer interface (BCI) recently demonstrated that deep neural networks (DNNs) can be effectively used for investigation of time sequences for physical actions (PA) classification. In this study, the relatively simple DNN with fully connected network (FCN) components and convolutional neural network (CNN) components was considered to classify finger-palm-hand manipulations each from the grasp-and-lift (GAL) dataset. The main aim of this study was to imitate and investigate environmental influence by the proposed noise data augmentation (NDA) of two kinds: (i) natural NDA by inclusion of noise EEG data from neighboring regions by increasing the sampling size N and the different offset values for sample labeling and (ii) synthetic NDA by adding the generated Gaussian noise. The natural NDA by increasing N leads to the higher micro and macro area under the curve (AUC) for receiver operating curve values for the bigger N values than usage of synthetic NDA. The detrended fluctuation analysis (DFA) was applied to investigate the fluctuation properties and calculate the correspondent Hurst exponents H for the quantitative characterization of the fluctuation variability. H values for the low time window scales (< 2 s) are higher in comparison with ones for the bigger time window scales. For example, H more than 2-3 times higher for some PAs, i.e., it means that the shorter EEG fragments (< 2 s) demonstrate the scaling behavior of the higher complexity than the longer fragments. As far as these results were obtained by the relatively small DNN with the low resource requirements, this approach can be promising for porting such models to Edge Computing infrastructures on devices with the very limited computational resources.

From Neural Networks to Emotional Networks: A Systematic Review of EEG-Based Emotion Recognition in Cognitive Neuroscience and Real-World Applications

BACKGROUND/OBJECTIVES

This systematic review presents how neural and emotional networks are integrated into EEG-based emotion recognition, bridging the gap between cognitive neuroscience and practical applications.

METHODS

Following PRISMA, 64 studies were reviewed that outlined the latest feature extraction and classification developments using deep learning models such as CNNs and RNNs.

RESULTS

Indeed, the findings showed that the multimodal approaches were practical, especially the combinations involving EEG with physiological signals, thus improving the accuracy of classification, even surpassing 90% in some studies. Key signal processing techniques used during this process include spectral features, connectivity analysis, and frontal asymmetry detection, which helped enhance the performance of recognition. Despite these advances, challenges remain more significant in real-time EEG processing, where a trade-off between accuracy and computational efficiency limits practical implementation. High computational cost is prohibitive to the use of deep learning models in real-world applications, therefore indicating a need for the development and application of optimization techniques. Aside from this, the significant obstacles are inconsistency in labeling emotions, variation in experimental protocols, and the use of non-standardized datasets regarding the generalizability of EEG-based emotion recognition systems.

DISCUSSION

These challenges include developing adaptive, real-time processing algorithms, integrating EEG with other inputs like facial expressions and physiological sensors, and a need for standardized protocols for emotion elicitation and classification. Further, related ethical issues with respect to privacy, data security, and machine learning model biases need to be much more proclaimed to responsibly apply research on emotions to areas such as healthcare, human-computer interaction, and marketing.

CONCLUSIONS

This review provides critical insight into and suggestions for further development in the field of EEG-based emotion recognition toward more robust, scalable, and ethical applications by consolidating current methodologies and identifying their key limitations.

A protocol for trustworthy EEG decoding with neural networks

Deep learning solutions have rapidly emerged for EEG decoding, achieving state-of-the-art performance on a variety of decoding tasks. Despite their high performance, existing solutions do not fully address the challenge posed by the introduction of many hyperparameters, defining data pre-processing, network architecture, network training, and data augmentation. Automatic hyperparameter search is rarely performed and limited to network-related hyperparameters. Moreover, pipelines are highly sensitive to performance fluctuations due to random initialization, hindering their reliability. Here, we design a comprehensive protocol for EEG decoding that explores the hyperparameters characterizing the entire pipeline and that includes multi-seed initialization for providing robust performance estimates. Our protocol is validated on 9 datasets about motor imagery, P300, SSVEP, including 204 participants and 26 recording sessions, and on different deep learning models. We accompany our protocol with extensive experiments on the main aspects influencing it, such as the number of participants used for hyperparameter search, the split into sequential simpler searches (multi-step search), the use of informed vs. non-informed search algorithms, and the number of random seeds for obtaining stable performance. The best protocol included 2-step hyperparameter search via an informed search algorithm, with the final training and evaluation performed using 10 random initializations. The optimal trade-off between performance and computational time was achieved by using a subset of 3-5 participants for hyperparameter search. Our protocol consistently outperformed baseline state-of-the-art pipelines, widely across datasets and models, and could represent a standard approach for neuroscientists for decoding EEG in a trustworthy and reliable way.

SpeechBrain-MOABB: An open-source Python library for benchmarking deep neural networks applied to EEG signals

Deep learning has revolutionized EEG decoding, showcasing its ability to outperform traditional machine learning models. However, unlike other fields, EEG decoding lacks comprehensive open-source libraries dedicated to neural networks. Existing tools (MOABB and braindecode) prevent the creation of robust and complete decoding pipelines, as they lack support for hyperparameter search across the entire pipeline, and are sensitive to fluctuations in results due to network random initialization. Furthermore, the absence of a standardized experimental protocol exacerbates the reproducibility crisis in the field. To address these limitations, we introduce SpeechBrain-MOABB, a novel open-source toolkit carefully designed to facilitate the development of a comprehensive EEG decoding pipeline based on deep learning. SpeechBrain-MOABB incorporates a complete experimental protocol that standardizes critical phases, such as hyperparameter search and model evaluation. It natively supports multi-step hyperparameter search for finding the optimal hyperparameters in a high-dimensional space defined by the entire pipeline, and multi-seed training and evaluation for obtaining performance estimates robust to the variability caused by random initialization. SpeechBrain-MOABB outperforms other libraries, including MOABB and braindecode, with accuracy improvements of 14.9% and 25.2% (on average), respectively. By enabling easy-to-use and easy-to-share decoding pipelines, our toolkit can be exploited by neuroscientists for decoding EEG with neural networks in a replicable and trustworthy way.

Altered Resting-State Electroencephalogram Microstate Characteristics in Stroke Patients

BACKGROUND

Stroke remains a leading cause of disability globally and movement impairment is the most common complication in stroke patients. Resting-state electroencephalography (EEG) microstate analysis is a non-invasive approach of whole-brain imaging based on the spatiotemporal pattern of the entire cerebral cortex. The present study aims to investigate microstate alterations in stroke patients.

METHODS

Resting-state EEG data collected from 24 stroke patients and 19 healthy controls matched by age and gender were subjected to microstate analysis. For four classic microstates labeled as class A, B, C and D, their temporal characteristics (duration, occurrence and coverage) and transition probabilities (TP) were extracted and compared between the two groups. Furthermore, we explored their correlations with clinical outcomes including the Fugl-Meyer assessment (FMA) and the action research arm test (ARAT) scores in stroke patients. Finally, we analyzed the relationship between the temporal characteristics and spectral power in frequency bands. False discovery rate (FDR) method was applied for correction of multiple comparisons.

RESULTS

Microstate analysis revealed that the stroke group had lower occurrence of microstate A which was regarded as the sensorimotor network (SMN) compared with the control group (p = 0.003, adjusted p = 0.036, t = -2.959). The TP from microstate A to microstate D had a significant positive correlation with the Fugl-Meyer assessment of lower extremity (FMA-LE) scores (p = 0.049, r = 0.406), but this finding did not survive FDR adjustment (adjusted p = 0.432). Additionally, the occurrence and the coverage of microstate B were negatively correlated with the power of delta band in the stroke group, which did not pass adjustment (p = 0.033, adjusted p = 0.790, r = -0.436; p = 0.026, adjusted p = 0.790, r = -0.454, respectively).

CONCLUSIONS

Our results confirm the abnormal temporal dynamics of brain activity in stroke patients. The study provides further electrophysiological evidence for understanding the mechanism of brain motor functional reorganization after stroke.

Simultaneous EEG-fNIRS Data Classification Through Selective Channel Representation and Spectrogram Imaging

The integration of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) can facilitate the advancement of brain-computer interfaces (BCIs). However, existing research in this domain has grappled with the challenge of the efficient selection of features, resulting in the underutilization of the temporal richness of EEG and the spatial specificity of fNIRS data.To effectively address this challenge, this study proposed a deep learning architecture called the multimodal DenseNet fusion (MDNF) model that was trained on two-dimensional (2D) EEG data images, leveraging advanced feature extraction techniques. The model transformed EEG data into 2D images using a short-time Fourier transform, applied transfer learning to extract discriminative features, and consequently integrated them with fNIRS-derived spectral entropy features. This approach aimed to bridge existing gaps in EEG-fNIRS-based BCI research by enhancing classification accuracy and versatility across various cognitive and motor imagery tasks.Experimental results on two public datasets demonstrated the superiority of our model over existing state-of-the-art methods.Thus, the high accuracy and precise feature utilization of the MDNF model demonstrates the potential in clinical applications for neurodiagnostics and rehabilitation, thereby paving the method for patient-specific therapeutic strategies.

EEG tensor decomposition delineates neurophysiological principles underlying conflict-modulated action restraint and action cancellation

Executive functions are essential for adaptive behavior. One executive function is the so-called 'interference control' or conflict monitoring another one is inhibitory control (i.e., action restraint and action cancelation). Recent evidence suggests an interplay of these processes, which is conceptually relevant given that newer conceptual frameworks imply that nominally different action/response control processes are explainable by a small set of cognitive and neurophysiological processes. The existence of such overarching neural principles has as yet not directly been examined. In the current study, we therefore use EEG tensor decomposition methods, to look into possible common neurophysiological signatures underlying conflict-modulated action restraint and action cancelation as mechanism underlying response inhibition. We show how conflicts differentially modulate action restraint and action cancelation processes and delineate common and distinct neural processes underlying this interplay. Concerning the spatial information modulations are similar in terms of an importance of processes reflected by parieto-occipital electrodes, suggesting that attentional selection processes play a role. Especially theta and alpha activity seem to play important roles. The data also show that tensor decomposition is sensitive to the manner of task implementation, thereby suggesting that switch probability/transitional probabilities should be taken into consideration when choosing tensor decomposition as analysis method. The study provides a blueprint of how to use tensor decomposition methods to delineate common and distinct neural mechanisms underlying action control functions using EEG data.

Response stopping under conflict: The integrative role of representational dynamics associated with the insular cortex

Coping with distracting inputs during goal-directed behavior is a common challenge, especially when stopping ongoing responses. The neural basis for this remains debated. Our study explores this using a conflict-modulation Stop Signal task, integrating group independent component analysis (group-ICA), multivariate pattern analysis (MVPA), and EEG source localization analysis. Consistent with previous findings, we show that stopping performance is better in congruent (nonconflicting) trials than in incongruent (conflicting) trials. Conflict effects in incongruent trials compromise stopping more due to the need for the reconfiguration of stimulus-response (S-R) mappings. These cognitive dynamics are reflected by four independent neural activity patterns (ICA), each coding representational content (MVPA). It is shown that each component was equally important in predicting behavioral outcomes. The data support an emerging idea that perception-action integration in action-stopping involves multiple independent neural activity patterns. One pattern relates to the precuneus (BA 7) and is involved in attention and early S-R processes. Of note, three other independent neural activity patterns were associated with the insular cortex (BA13) in distinct time windows. These patterns reflect a role in early attentional selection but also show the reiterated processing of representational content relevant for stopping in different S-R mapping contexts. Moreover, the insular cortex's role in automatic versus complex response selection in relation to stopping processes is shown. Overall, the insular cortex is depicted as a brain hub, crucial for response selection and cancellation across both straightforward (automatic) and complex (conditional) S-R mappings, providing a neural basis for general cognitive accounts on action control.

Self supervised learning based emotion recognition using physiological signals

Introduction

The significant role of emotional recognition in the field of human-machine interaction has garnered the attention of many researchers. Emotion recognition based on physiological signals can objectively reflect the most authentic emotional states of humans. However, existing labeled Electroencephalogram (EEG) datasets are often of small scale.

Methods

In practical scenarios, a large number of unlabeled EEG signals are easier to obtain. Therefore, this paper adopts self-supervised learning methods to study emotion recognition based on EEG. Specifically, experiments employ three pre-defined tasks to define pseudo-labels and extract features from the inherent structure of the data.

Results and discussion

Experimental results indicate that self-supervised learning methods have the capability to learn effective feature representations for downstream tasks without any manual labels.

Convolutional neural networks reveal properties of reach-to-grasp encoding in posterior parietal cortex

Deep neural networks (DNNs) are widely adopted to decode motor states from both non-invasively and invasively recorded neural signals, e.g., for realizing brain-computer interfaces. However, the neurophysiological interpretation of how DNNs make the decision based on the input neural activity is limitedly addressed, especially when applied to invasively recorded data. This reduces decoder reliability and transparency, and prevents the exploitation of decoders to better comprehend motor neural encoding. Here, we adopted an explainable artificial intelligence approach - based on a convolutional neural network and an explanation technique - to reveal spatial and temporal neural properties of reach-to-grasping from single-neuron recordings of the posterior parietal area V6A. The network was able to accurately decode 5 different grip types, and the explanation technique automatically identified the cells and temporal samples that most influenced the network prediction. Grip encoding in V6A neurons already started at movement preparation, peaking during movement execution. A difference was found within V6A: dorsal V6A neurons progressively encoded more for increasingly advanced grips, while ventral V6A neurons for increasingly rudimentary grips, with both subareas following a linear trend between the amount of grip encoding and the level of grip skills. By revealing the elements of the neural activity most relevant for each grip with no a priori assumptions, our approach supports and advances current knowledge about reach-to-grasp encoding in V6A, and it may represent a general tool able to investigate neural correlates of motor or cognitive tasks (e.g., attention and memory tasks) from single-neuron recordings.

An empirical comparison of deep learning explainability approaches for EEG using simulated ground truth

Recent advancements in machine learning and deep learning (DL) based neural decoders have significantly improved decoding capabilities using scalp electroencephalography (EEG). However, the interpretability of DL models remains an under-explored area. In this study, we compared multiple model explanation methods to identify the most suitable method for EEG and understand when some of these approaches might fail. A simulation framework was developed to evaluate the robustness and sensitivity of twelve back-propagation-based visualization methods by comparing to ground truth features. Multiple methods tested here showed reliability issues after randomizing either model weights or labels: e.g., the saliency approach, which is the most used visualization technique in EEG, was not class or model-specific. We found that DeepLift was consistently accurate as well as robust to detect the three key attributes tested here (temporal, spatial, and spectral precision). Overall, this study provides a review of model explanation methods for DL-based neural decoders and recommendations to understand when some of these methods fail and what they can capture in EEG.

Dissociable default-mode subnetworks subserve childhood attention and cognitive flexibility: Evidence from deep learning and stereotactic electroencephalography

Cognitive flexibility encompasses the ability to efficiently shift focus and forms a critical component of goal-directed attention. The neural substrates of this process are incompletely understood in part due to difficulties in sampling the involved circuitry. We leverage stereotactic intracranial recordings to directly resolve local-field potentials from otherwise inaccessible structures to study moment-to-moment attentional activity in children with epilepsy performing a flexible attentional task. On an individual subject level, we employed deep learning to decode neural features predictive of task performance indexed by single-trial reaction time. These models were subsequently aggregated across participants to identify predictive brain regions based on AAL atlas and FIND functional network parcellations. Through this approach, we show that fluctuations in beta (12-30 Hz) and gamma (30-80 Hz) power reflective of increased top-down attentional control and local neuronal processing within relevant large-scale networks can accurately predict single-trial task performance. We next performed connectomic profiling of these highly predictive nodes to examine task-related engagement of distributed functional networks, revealing exclusive recruitment of the dorsal default mode network during shifts in attention. The identification of distinct substreams within the default mode system supports a key role for this network in cognitive flexibility and attention in children. Furthermore, convergence of our results onto consistent functional networks despite significant inter-subject variability in electrode implantations supports a broader role for deep learning applied to intracranial electrodes in the study of human attention.

Decoding movement kinematics from EEG using an interpretable convolutional neural network

Continuous decoding of hand kinematics has been recently explored for the intuitive control of electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs). Deep neural networks (DNNs) are emerging as powerful decoders, for their ability to automatically learn features from lightly pre-processed signals. However, DNNs for kinematics decoding lack in the interpretability of the learned features and are only used to realize within-subject decoders without testing other training approaches potentially beneficial for reducing calibration time, such as transfer learning. Here, we aim to overcome these limitations by using an interpretable convolutional neural network (ICNN) to decode 2-D hand kinematics (position and velocity) from EEG in a pursuit tracking task performed by 13 participants. The ICNN is trained using both within-subject and cross-subject strategies, and also testing the feasibility of transferring the knowledge learned on other subjects on a new one. Moreover, the network eases the interpretation of learned spectral and spatial EEG features. Our ICNN outperformed most of the other state-of-the-art decoders, showing the best trade-off between performance, size, and training time. Furthermore, transfer learning improved kinematics prediction in the low data regime. The network attributed the highest relevance for decoding to the delta-band across all subjects, and to higher frequencies (alpha, beta, low-gamma) for a cluster of them; contralateral central and parieto-occipital sites were the most relevant, reflecting the involvement of sensorimotor, visual and visuo-motor processing. The approach improved the quality of kinematics prediction from the EEG, at the same time allowing interpretation of the most relevant spectral and spatial features.

Investigation of frequency components embedded in EEG recordings underlying neuronal mechanism of cognitive control and attentional functions

Attentional cognitive control regulates the perception to enhance human behaviour. The current study examines the atltentional mechanisms in terms of time and frequency of EEG signals. The cognitive load is higher for processing local attentional stimulus, thereby demanding higher response time (RT) with low response accuracy (RA). On the other hand, the global attentional mechanisms broadly promote the perception while demanding a low cognitive load with faster RT and high RA. Attentional mechanisms refer to perceptual systems that afford and allocate the adaptive behaviours for prioritizing the processing of relevant stimuli based on the local and global features. The early sensory component of C1, which was associated with the local attentional mechanism, showed higher amplitudes than the global attentional mechanisms in parieto-occipital regions. Further, the local attentional mechanisms were also sustained in N2 and P3 components increasing higher amplitude in the left and right hemispheric sides of temporal regions (T7 and T8). Theta band frequency had shown higher power spectrum density (PSD) values while processing local attentional mechanisms. However, the significance of other frequency bands was noticeably minute. Hence, integrating the attentional mechanisms in terms of ERP and frequency signatures, a hybrid custom weight allocation model (CWAM) was built to assess and predict the contribution of insignificant channels to significant ones. The CWAM model was formulated based on the computational linear regression derivatives. All the derivatives are computationally derived the significant score while channelizing the hierarchical performance of each channel with respect to the frequent and deviant occurrences of global-local stimulus. This model enables us to configure the neural dynamicity of cognitive allocation of resources within the different locations of the human brain while processing the attentional stimulus. CWAM is reported to be the first model to evaluate the performance of the non-significant channels for enhancing the response of significant channels. The findings of the CWAM model suggest that the brain's performance may be determined by the underlying contribution of the non-significant channels.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-022-09888-x.

Decoding neural activity to assess individual latent state in ecologically valid contexts

Objective.Currently, there exists very few ways to isolate cognitive processes, historically defined via highly controlled laboratory studies, in more ecologically valid contexts. Specifically, it remains unclear as to what extent patterns of neural activity observed under such constraints actually manifest outside the laboratory in a manner that can be used to make accurate inferences about latent states, associated cognitive processes, or proximal behavior. Improving our understanding of when and how specific patterns of neural activity manifest in ecologically valid scenarios would provide validation for laboratory-based approaches that study similar neural phenomena in isolation and meaningful insight into the latent states that occur during complex tasks.Approach.Domain generalization methods, borrowed from the work of the brain-computer interface community, have the potential to capture high-dimensional patterns of neural activity in a way that can be reliably applied across experimental datasets in order to address this specific challenge. We previously used such an approach to decode phasic neural responses associated with visual target discrimination. Here, we extend that work to more tonic phenomena such as internal latent states. We use data from two highly controlled laboratory paradigms to train two separate domain-generalized models. We apply the trained models to an ecologically valid paradigm in which participants performed multiple, concurrent driving-related tasks while perched atop a six-degrees-of-freedom ride-motion simulator.Main Results.Using the pretrained models, we estimate latent state and the associated patterns of neural activity. As the patterns of neural activity become more similar to those patterns observed in the training data, we find changes in behavior and task performance that are consistent with the observations from the original, laboratory-based paradigms.Significance.These results lend ecological validity to the original, highly controlled, experimental designs and provide a methodology for understanding the relationship between neural activity and behavior during complex tasks.

Event-related variability is modulated by task and development

In carefully designed experimental paradigms, cognitive scientists interpret the mean event-related potentials (ERP) in terms of cognitive operations. However, the huge signal variability from one trial to the next, questions the representability of such mean events. We explored here whether this variability is an unwanted noise, or an informative part of the neural response. We took advantage of the rapid changes in the visual system during human infancy and analyzed the variability of visual responses to central and lateralized faces in 2-to 6-month-old infants compared to adults using high-density electroencephalography (EEG). We observed that neural trajectories of individual trials always remain very far from ERP components, only moderately bending their direction with a substantial temporal jitter across trials. However, single trial trajectories displayed characteristic patterns of acceleration and deceleration when approaching ERP components, as if they were under the active influence of steering forces causing transient attraction and stabilization. These dynamic events could only partly be accounted for by induced microstate transitions or phase reset phenomena. Importantly, these structured modulations of response variability, both between and within trials, had a rich sequential organization, which in infants, was modulated by the task difficulty and age. Our approaches to characterize Event Related Variability (ERV) expand on classic ERP analyses and provide the first evidence for the functional role of ongoing neural variability in human infants.

Early categorization of social affordances during the visual encoding of bodily stimuli

Interpersonal interactions rely on various communication channels, both verbal and non-verbal, through which information regarding one's intentions and emotions are perceived. Here, we investigated the neural correlates underlying the visual processing of hand postures conveying social affordances (i.e., hand-shaking), compared to control stimuli such as hands performing non-social actions (i.e., grasping) or showing no movement at all. Combining univariate and multivariate analysis on electroencephalography (EEG) data, our results indicate that occipito-temporal electrodes show early differential processing of stimuli conveying social information compared to non-social ones. First, the amplitude of the Early Posterior Negativity (EPN, an Event-Related Potential related to the perception of body parts) is modulated differently during the perception of social and non-social content carried by hands. Moreover, our multivariate classification analysis (MultiVariate Pattern Analysis - MVPA) expanded the univariate results by revealing early (<200 ms) categorization of social affordances over occipito-parietal sites. In conclusion, we provide new evidence suggesting that the encoding of socially relevant hand gestures is categorized in the early stages of visual processing.

Characterization of attentional event-related potential from REM sleep behavior disorder patients based on explainable machine learning

BACKGROUND AND OBJECTIVE

Idiopathic rapid eye movement sleep behavior disorder (iRBD) is a prodromal stage of neurodegeneration and is associated with cortical dysfunction. The purpose of this study was to investigate the spatiotemporal characteristics of cortical activities underlying impaired visuospatial attention in iRBD patients using an explainable machine-learning approach.

METHODS

An algorithm based on a convolutional neural network (CNN) was devised to discriminate cortical current source activities of iRBD patients due to single-trial event-related potentials (ERPs), from those of normal controls. The ERPs from 16 iRBD patients and 19 age- and sex-matched normal controls were recorded while the subjects were performing visuospatial attentional task, and converted to two-dimensional images representing current source densities on flattened cortical surface. The CNN classifier was trained based on overall data, and then, a transfer learning approach was applied for the fine-tuning to each patient.

RESULTS

The trained classifier yielded high classification accuracy. The critical features for the classification were determined by layer-wise relevance propagation, so that the spatiotemporal characteristics of cortical activities that were most relevant to cognitive impairment in iRBD were revealed.

CONCLUSIONS

These results suggest that the recognized dysfunction in visuospatial attention of iRBD patients originates from neural activity impairment in relevant cortical regions and may contribute to the development of useful iRBD biomarkers based on neural activity.

Classification of age groups and task conditions provides additional evidence for differences in electrophysiological correlates of inhibitory control across the lifespan

The aim of this study was to extend previous findings on selective attention over a lifetime using machine learning procedures. By decoding group membership and stimulus type, we aimed to study differences in the neural representation of inhibitory control across age groups at a single-trial level. We re-analyzed data from 211 subjects from six age groups between 8 and 83 years of age. Based on single-trial EEG recordings during a flanker task, we used support vector machines to predict the age group as well as to determine the presented stimulus type (i.e., congruent, or incongruent stimulus). The classification of group membership was highly above chance level (accuracy: 55%, chance level: 17%). Early EEG responses were found to play an important role, and a grouped pattern of classification performance emerged corresponding to age structure. There was a clear cluster of individuals after retirement, i.e., misclassifications mostly occurred within this cluster. The stimulus type could be classified above chance level in ~ 95% of subjects. We identified time windows relevant for classification performance that are discussed in the context of early visual attention and conflict processing. In children and older adults, a high variability and latency of these time windows were found. We were able to demonstrate differences in neuronal dynamics at the level of individual trials. Our analysis was sensitive to mapping gross changes, e.g., at retirement age, and to differentiating components of visual attention across age groups, adding value for the diagnosis of cognitive status across the lifespan. Overall, the results highlight the use of machine learning in the study of brain activity over a lifetime.

Deep learning applied to EEG source-data reveals both ventral and dorsal visual stream involvement in holistic processing of social stimuli

Perception of social stimuli (faces and bodies) relies on "holistic" (i.e., global) mechanisms, as supported by picture-plane inversion: perceiving inverted faces/bodies is harder than perceiving their upright counterpart. Albeit neuroimaging evidence suggested involvement of face-specific brain areas in holistic processing, their spatiotemporal dynamics and selectivity for social stimuli is still debated. Here, we investigate the spatiotemporal dynamics of holistic processing for faces, bodies and houses (adopted as control non-social category), by applying deep learning to high-density electroencephalographic signals (EEG) at source-level. Convolutional neural networks were trained to classify cortical EEG responses to stimulus orientation (upright/inverted), separately for each stimulus type (faces, bodies, houses), resulting to perform well above chance for faces and bodies, and close to chance for houses. By explaining network decision, the 150-200 ms time interval and few visual ventral-stream regions were identified as mostly relevant for discriminating face and body orientation (lateral occipital cortex, and for face only, precuneus cortex, fusiform and lingual gyri), together with two additional dorsal-stream areas (superior and inferior parietal cortices). Overall, the proposed approach is sensitive in detecting cortical activity underlying perceptual phenomena, and by maximally exploiting discriminant information contained in data, may reveal spatiotemporal features previously undisclosed, stimulating novel investigations.

Energy-Efficient EEG-Based Scheme for Autism Spectrum Disorder Detection Using Wearable Sensors

The deployment of wearable wireless systems that collect physiological indicators to aid in diagnosing neurological disorders represents a potential solution for the new generation of e-health systems. Electroencephalography (EEG), a recording of the brain's electrical activity, is a promising physiological test for the diagnosis of autism spectrum disorders. It can identify the abnormalities of the neural system that are associated with autism spectrum disorders. However, streaming EEG samples remotely for classification can reduce the wireless sensor's lifespan and creates doubt regarding the application's feasibility. Therefore, decreasing data transmission may conserve sensor energy and extend the lifespan of wireless sensor networks. This paper suggests the development of a sensor-based scheme for early age autism detection. The proposed scheme implements an energy-efficient method for signal transformation allowing relevant feature extraction for accurate classification using machine learning algorithms. The experimental results indicate an accuracy of 96%, a sensitivity of 100%, and around 95% of F1 score for all used machine learning models. The results also show that our scheme energy consumption is 97% lower than streaming the raw EEG samples.

Optimization of data pre-processing methods for time-series classification of electroencephalography data

The performance of time-series classification of electroencephalographic data varies strongly across experimental paradigms and study participants. Reasons are task-dependent differences in neuronal processing and seemingly random variations between subjects, amongst others. The effect of data pre-processing techniques to ameliorate these challenges is relatively little studied. Here, the influence of spatial filter optimization methods and non-linear data transformation on time-series classification performance is analyzed by the example of high-frequency somatosensory evoked responses. This is a model paradigm for the analysis of high-frequency electroencephalography data at a very low signal-to-noise ratio, which emphasizes the differences of the explored methods. For the utilized data, it was found that the individual signal-to-noise ratio explained up to 74% of the performance differences between subjects. While data pre-processing was shown to increase average time-series classification performance, it could not fully compensate the signal-to-noise ratio differences between the subjects. This study proposes an algorithm to prototype and benchmark pre-processing pipelines for a paradigm and data set at hand. Extreme learning machines, Random Forest, and Logistic Regression can be used quickly to compare a set of potentially suitable pipelines. For subsequent classification, however, machine learning models were shown to provide better accuracy.

On the Role of Stimulus-Response Context in Inhibitory Control in Alcohol Use Disorder

The behavioral and neural dynamics of response inhibition deficits in alcohol use disorder (AUD) are still largely unclear, despite them possibly being key to the mechanistic understanding of the disorder. Our study investigated the effect of automatic vs. controlled processing during response inhibition in participants with mild-to-moderate AUD and matched healthy controls. For this, a Simon Nogo task was combined with EEG signal decomposition, multivariate pattern analysis (MVPA), and source localization methods. The final sample comprised n = 59 (32♂) AUD participants and n = 64 (28♂) control participants. Compared with the control group, AUD participants showed overall better response inhibition performance. Furthermore, the AUD group was less influenced by the modulatory effect of automatic vs. controlled processes during response inhibition (i.e., had a smaller Simon Nogo effect). The neurophysiological data revealed that the reduced Simon Nogo effect in the AUD group was associated with reduced activation differences between congruent and incongruent Nogo trials in the inferior and middle frontal gyrus. Notably, the drinking frequency (but not the number of AUD criteria we had used to distinguish groups) predicted the extent of the Simon Nogo effect. We suggest that the counterintuitive advantage of participants with mild-to-moderate AUD over those in the control group could be explained by the allostatic model of drinking effects.

Evidence for independent representational contents in inhibitory control subprocesses associated with frontoparietal cortices

Inhibitory control processes have intensively been studied in cognitive science for the past decades. Even though the neural dynamics underlying these processes are increasingly better understood, a critical open question is how the representational dynamics of the inhibitory control processes are modulated when engaging in response inhibition in a relatively automatic or a controlled mode. Against the background of an overarching theory of perception-action integration, we combine temporal and spatial EEG signal decomposition methods with multivariate pattern analysis and source localization to obtain fine-grained insights into the neural dynamics of the representational content of response inhibition. For this purpose, we used a sample of N = 40 healthy adult participants. The behavioural data suggest that response inhibition was better in a more controlled than a more automated response execution mode. Regarding neural dynamics, effects of response inhibition modes relied on a concomitant coding of stimulus-related information and rules of how stimulus information is related to the appropriate motor programme. Crucially, these fractions of information, which are encoded at the same time in the neurophysiological signal, are based on two independent spatial neurophysiological activity patterns, also showing differences in the temporal stability of the representational content. Source localizations revealed that the precuneus and inferior parietal cortex regions are more relevant than prefrontal areas for the representation of stimulus-response selection codes. We provide a blueprint how a concatenation of EEG signal analysis methods, capturing distinct aspects of neural dynamics, can be connected to cognitive science theory on the importance of representations in action control.

Explainable Machine-Learning-Based Characterization of Abnormal Cortical Activities for Working Memory of Restless Legs Syndrome Patients

Restless legs syndrome (RLS) is a sensorimotor disorder accompanied by a strong urge to move the legs and an unpleasant sensation in the legs, and is known to accompany prefrontal dysfunction. Here, we aimed to clarify the neural mechanism of working memory deficits associated with RLS using machine-learning-based analysis of single-trial neural activities. A convolutional neural network classifier was developed to discriminate the cortical activities between RLS patients and normal controls. A layer-wise relevance propagation was applied to the trained classifier in order to determine the critical nodes in the input layer for the output decision, i.e., the time/location of cortical activities discriminating RLS patients and normal controls during working memory tasks. Our method provided high classification accuracy (~94%) from single-trial event-related potentials, which are known to suffer from high inter-trial/inter-subject variation and low signal-to-noise ratio, after strict separation of training/test/validation data according to leave-one-subject-out cross-validation. The determined critical areas overlapped with the cortical substrates of working memory, and the neural activities in these areas were correlated with some significant clinical scores of RLS.

A Model Visualization-based Approach for Insight into Waveforms and Spectra Learned by CNNs

Recent years have shown a growth in the application of deep learning architectures such as convolutional neural networks (CNNs), to electrophysiology analysis. However, using neural networks with raw time-series data makes explainability a significant challenge. Multiple explainability approaches have been developed for insight into the spectral features learned by CNNs from EEG. However, across electrophysiology modalities, and even within EEG, there are many unique waveforms of clinical relevance. Existing methods that provide insight into waveforms learned by CNNs are of questionable utility. In this study, we present a novel model visualization-based approach that analyzes the filters in the first convolutional layer of the network. To our knowledge, this is the first method focused on extracting explainable information from EEG waveforms learned by CNNs while also providing insight into the learned spectral features. We demonstrate the viability of our approach within the context of automated sleep stage classification, a well-characterized domain that can help validate our approach. We identify 3 subgroups of filters with distinct spectral properties, determine the relative importance of each group of filters, and identify several unique waveforms learned by the classifier that were vital to the classifier performance. Our approach represents a significant step forward in explainability for electrophysiology classifiers, which we also hope will be useful for providing insights in future studies. Clinical Relevance- Our approach can assist with the development and validation of clinical time-series classifiers.

Facial Emotions Are Accurately Encoded in the Neural Signal of Those With Autism Spectrum Disorder: A Deep Learning Approach

BACKGROUND

Individuals with autism spectrum disorder (ASD) exhibit frequent behavioral deficits in facial emotion recognition (FER). It remains unknown whether these deficits arise because facial emotion information is not encoded in their neural signal or because it is encodes but fails to translate to FER behavior (deployment). This distinction has functional implications, including constraining when differences in social information processing occur in ASD, and guiding interventions (i.e., developing prosthetic FER vs. reinforcing existing skills).

METHODS

We utilized a discriminative and contemporary machine learning approach-deep convolutional neural networks-to classify facial emotions viewed by individuals with and without ASD (N = 88) from concurrently recorded electroencephalography signals.

RESULTS

The convolutional neural network classified facial emotions with high accuracy for both ASD and non-ASD groups, even though individuals with ASD performed more poorly on the concurrent FER task. In fact, convolutional neural network accuracy was greater in the ASD group and was not related to behavioral performance. This pattern of results replicated across three independent participant samples. Moreover, feature importance analyses suggested that a late temporal window of neural activity (1000-1500 ms) may be uniquely important in facial emotion classification for individuals with ASD.

CONCLUSIONS

Our results reveal for the first time that facial emotion information is encoded in the neural signal of individuals with (and without) ASD. Thus, observed difficulties in behavioral FER associated with ASD likely arise from difficulties in decoding or deployment of facial emotion information within the neural signal. Interventions should focus on capitalizing on this intact encoding rather than promoting compensation or FER prostheses.

A Bayesian-optimized design for an interpretable convolutional neural network to decode and analyze the P300 response in autism

OBJECTIVE

P300 can be analyzed in autism spectrum disorder (ASD) to derive biomarkers and can be decoded in BCIs to reinforce ASD impaired skills. Convolutional neural networks (CNNs) have been proposed for P300 decoding, outperforming traditional algorithms but they i) do not investigate optimal designs in different training conditions; ii) lack in interpretability. To overcome these limitations, an interpretable CNN (ICNN), that we recently proposed for motor decoding, has been modified and adopted here, with its optimal design searched via Bayesian optimization.

APPROACH

The ICNN provides a straightforward interpretation of spectral and spatial features learned to decode P300. The Bayesian-optimized (BO) ICNN design was investigated separately for different training strategies (within-subject, within-session, and cross-subject) and BO models were used for the subsequent analyses. Specifically, transfer learning (TL) potentialities were investigated by assessing how pretrained cross-subject BO models performed on a new subject vs. random-initialized models. Furthermore, within-subject BO-derived models were combined with an Explanation Technique (ICNN+ET) to analyze P300 spectral and spatial features.

MAIN RESULTS

The ICNN resulted comparable or even outperformed existing CNNs, at the same time being lighter. Bayesian-optimized ICNN designs differed depending on the training strategy, needing more capacity as the training set variability increased. Furthermore, TL provided higher performance than networks trained from scratch. The ICNN+ET analysis suggested the frequency range [2, 5.8] Hz as the most relevant, and spatial features showed a right-hemispheric parietal asymmetry. The ICNN+ET-derived features, but not ERP-derived features, resulted significantly and highly correlated to ADOS clinical scores.

SIGNIFICANCE

This study substantiates the idea that a CNN can be designed both accurate and interpretable for P300 decoding, with an optimized design depending on the training condition. The novel ICNN-based analysis tool was able to better capture ASD neural signatures than traditional ERP analysis, possibly paving the way for identifying novel biomarkers.

IENet: a robust convolutional neural network for EEG based brain-computer interfaces

OBJECTIVE

Brain-computer interfaces (BCIs) based on electroencephalogram (EEG) develop into novel application areas with more complex scenarios, which put forward higher requirements for the robustness of EEG signal processing algorithms. Deep learning can automatically extract discriminative features and potential dependencies via deep structures, demonstrating strong analytical capabilities in numerous domains such as computer vision (CV) and natural language processing (NLP). Making full use of deep learning technology to design a robust algorithm that is capable of analyzing EEG across BCI paradigms is our main work in this paper.

APPROACH

Inspired by InceptionV4 and InceptionTime architecture, we introduce a neural network ensemble named InceptionEEG-Net (IENet), where multi-scale convolutional layer and convolution of length 1 enable model to extract rich high-dimensional features with limited parameters. In addition, we propose the average receptive field gain for convolutional neural networks (CNNs), which optimizes IENet to detect long patterns at a smaller cost. We compare with the current state-of-the-art method across five EEG-BCI paradigms: steady-state visual evoked potentials, epilepsy EEG, overt attention P300 visual-evoked potentials, covert attention P300 visual-evoked potentials and movement-related cortical potentials.

MAIN RESULTS

The classification results show that the generalizability of IENet is on par with the state-of-the-art paradigm-agnostic models on test datasets. Furthermore, the feature explainability analysis of IENet illustrates its capability to extract neurophysiologically interpretable features for different BCI paradigms, ensuring the reliability of algorithm. Significance. It can be seen from our results that IENet can generalize to different BCI paradigms. And it is essential for deep CNNs to increase the receptive field size using average receptive field gain.

The Nencki-Symfonia electroencephalography/event-related potential dataset: Multiple cognitive tasks and resting-state data collected in a sample of healthy adults

BACKGROUND

One of the goals of neuropsychology is to understand the brain mechanisms underlying aspects of attention and cognitive control. Several tasks have been developed as a part of this body of research, however their results are not always consistent. A reliable comparison of the data and a synthesis of study conclusions has been precluded by multiple methodological differences. Here, we describe a publicly available, high-density electroencephalography (EEG) dataset obtained from 42 healthy young adults while they performed 3 cognitive tasks: (i) an extended multi-source interference task; (ii) a 3-stimuli oddball task; (iii) a control, simple reaction task; and (iv) a resting-state protocol. Demographic and psychometric information are included within the dataset.

DATASET VALIDATION

First, data validation confirmed acceptable quality of the obtained EEG signals. Typical event-related potential (ERP) waveforms were obtained, as expected for attention and cognitive control tasks (i.e., N200, P300, N450). Behavioral results showed the expected progression of reaction times and error rates, which confirmed the effectiveness of the applied paradigms.

CONCLUSIONS

This dataset is well suited for neuropsychological research regarding common and distinct mechanisms involved in different cognitive tasks. Using this dataset, researchers can compare a wide range of classical EEG/ERP features across tasks for any selected subset of electrodes. At the same time, 128-channel EEG recording allows for source localization and detailed connectivity studies. Neurophysiological measures can be correlated with additional psychometric data obtained from the same participants. This dataset can also be used to develop and verify novel analytical and classification approaches that can advance the field of deep/machine learning algorithms, recognition of single-trial ERP responses to different task conditions, and detection of EEG/ERP features for use in brain-computer interface applications.

Pretrial Theta Band Activity Affects Context-dependent Modulation of Response Inhibition

The ability to inhibit a prepotent response is a crucial prerequisite of goal-directed behavior. So far, research on response inhibition has mainly examined these processes when there is little to no cognitive control during the decision to respond. We manipulated the "context" in which response inhibition has to be exerted (i.e., a controlled or an automated context) by combining a Simon task with a go/no-go task and focused on theta band activity. To investigate the role of "context" in response inhibition, we also examined how far theta band activity in the pretrial period modulates context-dependent variations of theta band activity during response inhibition. This was done in an EEG study applying beamforming methods. Here, we examined n = 43 individuals. We show that an automated context, as opposed to a controlled context, compromises response inhibition performance and increases the need for cognitive control. This was also related to context-dependent modulations of theta band activity in superior frontal and middle frontal regions. Of note, results showed that theta band activity in the pretrial period, associated with the right inferior frontal cortex, was substantially correlated with context-dependent modulations of theta band activity during response inhibition. The direction of the obtained correlation provides insights into the functional relevance of a pretrial theta band activity. The data suggest that pretrial theta band activity reflects some form of attentional sampling to inform possible upcoming processes signaling the need for cognitive control.

EEG-Based Emotion Recognition Using Deep Learning and M3GP

This paper presents the proposal of a method to recognize emotional states through EEG analysis. The novelty of this work lies in its feature improvement strategy, based on multiclass genetic programming with multidimensional populations (M3GP), which builds features by implementing an evolutionary technique that selects, combines, deletes, and constructs the most suitable features to ease the classification process of the learning method. In this way, the problem data can be mapped into a more favorable search space that best defines each class. After implementing the M3GP, the results showed an increment of 14.76% in the recognition rate without changing any settings in the learning method. The tests were performed on a biometric EEG dataset (BED), designed to evoke emotions and record the cerebral cortex’s electrical response; this dataset implements a low cost device to collect the EEG signals, allowing greater viability for the application of the results. The proposed methodology achieves a mean classification rate of 92.1%, and simplifies the feature management process by increasing the separability of the spectral features.

Use of an Artificial Neural Network for Tensile Strength Prediction of Nano Titanium Dioxide Coated Cotton

In this study, an artificial neural network (ANN) is used for the prediction of tensile strength of nano titanium dioxide (TiO₂) coated cotton. The coating process was performed by ultraviolet (UV) radiations. Later on, a backpropagation ANN algorithm trained with Bayesian regularization was applied to predict the tensile strength. For a comparative study, ANN results were compared with traditional methods including multiple linear regression (MLR) and polynomial regression analysis (PRA). The input conditions for the experiment were dosage of TiO₂, UV irradiation time and temperature of the system. Simulation results elucidated that ANN model provides high performance accuracy than MLR and PRA. In addition, statistical analysis was also performed to check the significance of this study. The results show a strong correlation between predicted and measured tensile strength of nano TiO2-coated cotton with small error values.

Conditional generative adversarial networks applied to EEG data can inform about the inter-relation of antagonistic behaviors on a neural level

Goal-directed actions frequently require a balance between antagonistic processes (e.g., executing and inhibiting a response), often showing an interdependency concerning what constitutes goal-directed behavior. While an inter-dependency of antagonistic actions is well described at a behavioral level, a possible inter-dependency of underlying processes at a neuronal level is still enigmatic. However, if there is an interdependency, it should be possible to predict the neurophysiological processes underlying inhibitory control based on the neural processes underlying speeded automatic responses. Based on that rationale, we applied artificial intelligence and source localization methods to human EEG recordings from N = 255 participants undergoing a response inhibition experiment (Go/Nogo task). We show that the amplitude and timing of scalp potentials and their functional neuroanatomical sources during inhibitory control can be inferred by conditional generative adversarial networks (cGANs) using neurophysiological data recorded during response execution. We provide insights into possible limitations in the use of cGANs to delineate the interdependency of antagonistic actions on a neurophysiological level. Nevertheless, artificial intelligence methods can provide information about interdependencies between opposing cognitive processes on a neurophysiological level with relevance for cognitive theory.

Convolutional neural networks for decoding electroencephalography responses and visualizing trial by trial changes in discriminant features

BACKGROUND

Deep learning has revolutionized the field of computer vision, where convolutional neural networks (CNNs) extract complex patterns of information from large datasets. The use of deep networks in neuroscience is mainly focused to neuroimaging or brain computer interface -BCI- applications. In electroencephalography (EEG) research, multivariate pattern analysis (MVPA) mainly relies on linear algorithms, which require a homogeneous dataset and assume that discriminant features appear at consistent latencies and electrodes across trials. However, neural responses may shift in time or space during an experiment, resulting in under-estimation of discriminant features. Here, we aimed at using CNNs to classify EEG responses to external stimuli, by taking advantage of time- and space- unlocked neural activity, and at examining how discriminant features change over the course of an experiment, on a trial by trial basis.

NEW METHOD

We present a novel pipeline, consisting of data augmentation, CNN training, and feature visualization techniques, fine-tuned for MVPA on EEG data.

RESULTS

Our pipeline provides high classification performance and generalizes to new datasets. Additionally, we show that the features identified by the CNN for classification are electrophysiologically interpretable and can be reconstructed at the single-trial level to study trial-by-trial evolution of class-specific discriminant activity.

COMPARISON WITH EXISTING TECHNIQUES

The developed pipeline was compared to commonly used MVPA algorithms like logistic regression and support vector machines, as well as to shallow and deep convolutional neural networks. Our approach yielded significantly higher classification performance than existing MVPA techniques (p = 0.006) and comparable results to other CNNs for EEG data.

CONCLUSION

In summary, we present a novel deep learning pipeline for MVPA of EEG data, that can extract trial-by-trial discriminative activity in a data-driven way.

Categorizing objects from MEG signals using EEGNet

Magnetoencephalography (MEG) signals have demonstrated their practical application to reading human minds. Current neural decoding studies have made great progress to build subject-wise decoding models to extract and discriminate the temporal/spatial features in neural signals. In this paper, we used a compact convolutional neural network-EEGNet-to build a common decoder across subjects, which deciphered the categories of objects (faces, tools, animals, and scenes) from MEG data. This study investigated the influence of the spatiotemporal structure of MEG on EEGNet's classification performance. Furthermore, the EEGNet replaced its convolution layers with two sets of parallel convolution structures to extract the spatial and temporal features simultaneously. Our results showed that the organization of MEG data fed into the EEGNet has an effect on EEGNet classification accuracy, and the parallel convolution structures in EEGNet are beneficial to extracting and fusing spatial and temporal MEG features. The classification accuracy demonstrated that the EEGNet succeeds in building the common decoder model across subjects, and outperforms several state-of-the-art feature fusing methods.

Prediction of Methylene Blue Removal by Nano TiO2 Using Deep Neural Network

This paper deals with the prediction of methylene blue (MB) dye removal under the influence of titanium dioxide nanoparticles (TiO2 NPs) through deep neural network (DNN). In the first step, TiO2 NPs were prepared and their morphological properties were analysed by scanning electron microscopy. Later, the influence of as synthesized TiO2 NPs was tested against MB dye removal and in the final step, DNN was used for the prediction. DNN is an efficient machine learning tools and widely used model for the prediction of highly complex problems. However, it has never been used for the prediction of MB dye removal. Therefore, this paper investigates the prediction accuracy of MB dye removal under the influence of TiO2 NPs using DNN. Furthermore, the proposed DNN model was used to map out the complex input-output conditions for the prediction of optimal results. The amount of chemicals, i.e., amount of TiO2 NPs, amount of ehylene glycol and reaction time were chosen as input variables and MB dye removal percentage was evaluated as a response. DNN model provides significantly high performance accuracy for the prediction of MB dye removal and can be used as a powerful tool for the prediction of other functional properties of nanocomposites.

Universum based Lagrangian twin bounded support vector machine to classify EEG signals

BACKGROUND AND OBJECTIVE

The detection of brain-related problems and neurological disorders like epilepsy, sleep disorder, and so on is done by using electroencephalogram (EEG) signals which contain noisy signals and outliers. Universum data contains a set of a sample that does not belong to any of the concerned classes and serves as the advanced knowledge about the data distribution. Earlier information has been utilized viably in improving classification performance. Recently a novel universum support vector machine (USVM) was proposed for EEG signal classification and further, a universum twin support vector machine (UTWSVM) was proposed based on USVM to improve the performance. Inspired by USVM and UTWSVM, this paper suggests a novel method called universum based Lagrangian twin bounded support vector machine (ULTBSVM), where universum data is utilized to incorporate the prior information about the data distribution to classify healthy and seizure EEG signals.

METHODS

In the proposed ULTBSVM the square of the 2-norm of the slack variables is used to formulate the objective function strongly convex; hence it always gives unique solutions. Unlike twin support vector machine (TWSVM) and universum twin support vector machine (UTWSVM), the proposed ULTBSVM is having regularization terms that follow the structural risk minimization (SRM) principle and enhance the stability in the dual formulations, make the model well-posed and prevents the overfitting problem. Here, interracial EEG data have been considered as universum data to classify healthy and seizure signals. Several feature extraction techniques have been implemented to get important noiseless features.

RESULTS

Several EEG datasets, as well as publicly available UCI datasets, are utilized to assess the performance of the proposed method. An analytical comparison has been performed of the proposed method with USVM and UTWSVM to detect seizure and healthy signals and for real-world data, the ULTBSVM is compared with the universum based models as well as TWSVM and the proposed method gives better results in most of the cases as compared to the other methods.

CONCLUSION

The results clearly show that ULTBSVM is a potential method for the classification of EEG signals as well as real-world datasets having interracial data as universum data. Here we have used universum points for the binary class classification problem, but one can extend and use it for multi-class classification problems as well.

Emotion Recognition by Correlating Facial Expressions and EEG Analysis

Emotion recognition is a fundamental task that any affective computing system must perform to adapt to the user’s current mood. The analysis of electroencephalography signals has gained notoriety in studying human emotions because of its non-invasive nature. This paper presents a two-stage deep learning model to recognize emotional states by correlating facial expressions and brain signals. Most of the works related to the analysis of emotional states are based on analyzing large segments of signals, generally as long as the evoked potential lasts, which could cause many other phenomena to be involved in the recognition process. Unlike with other phenomena, such as epilepsy, there is no clearly defined marker of when an event begins or ends. The novelty of the proposed model resides in the use of facial expressions as markers to improve the recognition process. This work uses a facial emotion recognition technique (FER) to create identifiers each time an emotional response is detected and uses them to extract segments of electroencephalography (EEG) records that a priori will be considered relevant for the analysis. The proposed model was tested on the DEAP dataset.

Evaluating deep learning EEG-based mental stress classification in adolescents with autism for breathing entrainment BCI

Mental stress is a major individual and societal burden and one of the main contributing factors that lead to pathologies such as depression, anxiety disorders, heart attacks, and strokes. Given that anxiety disorders are one of the most common comorbidities in youth with autism spectrum disorder (ASD), this population is particularly vulnerable to mental stress, severely limiting overall quality of life. To prevent this, early stress quantification with machine learning (ML) and effective anxiety mitigation with non-pharmacological interventions are essential. This study aims to investigate the feasibility of exploiting electroencephalography (EEG) signals for stress assessment by comparing several ML classifiers, namely support vector machine (SVM) and deep learning methods. We trained a total of eleven subject-dependent models-four with conventional brain-computer interface (BCI) methods and seven with deep learning approaches-on the EEG of neurotypical (n=5) and ASD (n=8) participants performing alternating blocks of mental arithmetic stress induction, guided and unguided breathing. Our results show that a multiclass two-layer LSTM RNN deep learning classifier is capable of identifying mental stress from ongoing EEG with an overall accuracy of 93.27%. Our study is the first to successfully apply an LSTM RNN classifier to identify stress states from EEG in both ASD and neurotypical adolescents, and offers promise for an EEG-based BCI for the real-time assessment and mitigation of mental stress through a closed-loop adaptation of respiration entrainment.

A Lightweight Multi-Scale Convolutional Neural Network for P300 Decoding: Analysis of Training Strategies and Uncovering of Network Decision

Convolutional neural networks (CNNs), which automatically learn features from raw data to approximate functions, are being increasingly applied to the end-to-end analysis of electroencephalographic (EEG) signals, especially for decoding brain states in brain-computer interfaces (BCIs). Nevertheless, CNNs introduce a large number of trainable parameters, may require long training times, and lack in interpretability of learned features. The aim of this study is to propose a CNN design for P300 decoding with emphasis on its lightweight design while guaranteeing high performance, on the effects of different training strategies, and on the use of post-hoc techniques to explain network decisions. The proposed design, named MS-EEGNet, learned temporal features in two different timescales (i.e., multi-scale, MS) in an efficient and optimized (in terms of trainable parameters) way, and was validated on three P300 datasets. The CNN was trained using different strategies (within-participant and within-session, within-participant and cross-session, leave-one-subject-out, transfer learning) and was compared with several state-of-the-art (SOA) algorithms. Furthermore, variants of the baseline MS-EEGNet were analyzed to evaluate the impact of different hyper-parameters on performance. Lastly, saliency maps were used to derive representations of the relevant spatio-temporal features that drove CNN decisions. MS-EEGNet was the lightest CNN compared with the tested SOA CNNs, despite its multiple timescales, and significantly outperformed the SOA algorithms. Post-hoc hyper-parameter analysis confirmed the benefits of the innovative aspects of MS-EEGNet. Furthermore, MS-EEGNet did benefit from transfer learning, especially using a low number of training examples, suggesting that the proposed approach could be used in BCIs to accurately decode the P300 event while reducing calibration times. Representations derived from the saliency maps matched the P300 spatio-temporal distribution, further validating the proposed decoding approach. This study, by specifically addressing the aspects of lightweight design, transfer learning, and interpretability, can contribute to advance the development of deep learning algorithms for P300-based BCIs.

Evaluation of Three Machine Learning Algorithms for the Automatic Classification of EMG Patterns in Gait Disorders

Gait disorders are common in neurodegenerative diseases and distinguishing between seemingly similar kinematic patterns associated with different pathological entities is a challenge even for the experienced clinician. Ultimately, muscle activity underlies the generation of kinematic patterns. Therefore, one possible way to address this problem may be to differentiate gait disorders by analyzing intrinsic features of muscle activations patterns. Here, we examined whether it is possible to differentiate electromyography (EMG) gait patterns of healthy subjects and patients with different gait disorders using machine learning techniques. Nineteen healthy volunteers (9 male, 10 female, age 28.2 ± 6.2 years) and 18 patients with gait disorders (10 male, 8 female, age 66.2 ± 14.7 years) resulting from different neurological diseases walked down a hallway 10 times at a convenient pace while their muscle activity was recorded via surface EMG electrodes attached to 5 muscles of each leg (10 channels in total). Gait disorders were classified as predominantly hypokinetic (n = 12) or ataxic (n = 6) gait by two experienced raters based on video recordings. Three different classification methods (Convolutional Neural Network-CNN, Support Vector Machine-SVM, K-Nearest Neighbors-KNN) were used to automatically classify EMG patterns according to the underlying gait disorder and differentiate patients and healthy participants. Using a leave-one-out approach for training and evaluating the classifiers, the automatic classification of normal and abnormal EMG patterns during gait (2 classes: "healthy" and "patient") was possible with a high degree of accuracy using CNN (accuracy 91.9%), but not SVM (accuracy 67.6%) or KNN (accuracy 48.7%). For classification of hypokinetic vs. ataxic vs. normal gait (3 classes) best results were again obtained for CNN (accuracy 83.8%) while SVM and KNN performed worse (accuracy SVM 51.4%, KNN 32.4%). These results suggest that machine learning methods are useful for distinguishing individuals with gait disorders from healthy controls and may help classification with respect to the underlying disorder even when classifiers are trained on comparably small cohorts. In our study, CNN achieved higher accuracy than SVM and KNN and may constitute a promising method for further investigation.

Anodal tDCS modulates specific processing codes during conflict monitoring associated with superior and middle frontal cortices

Conflict monitoring processes are central for cognitive control. Neurophysiological correlates of conflict monitoring (i.e. the N2 ERP) likely represent a mixture of different cognitive processes. Based on theoretical considerations, we hypothesized that effects of anodal tDCS (atDCS) in superior frontal areas affect specific subprocesses in neurophysiological activity during conflict monitoring. To investigate this, young healthy adults performed a Simon task while EEG was recorded. atDCS and sham tDCS were applied in a single-blind, cross-over study design. Using temporal signal decomposition in combination with source localization analyses, we demonstrated that atDCS effects on cognitive control are very specific: the detrimental effect of atDCS on response speed was largest in case of response conflicts. This however only showed in aspects of the decomposed N2 component, reflecting stimulus-response translation processes. In contrast to this, stimulus-related aspects of the N2 as well as purely response-related processes were not modulated by atDCS. EEG source localization analyses revealed that the effect was likely driven by activity modulations in the superior frontal areas, including the supplementary motor cortex (BA6), as well as middle frontal (BA9) and medial frontal areas (BA32). atDCS did not modulate effects of proprioceptive information on hand position, even though this aspect is known to be processed within the same brain areas. Physiological effects of atDCS likely modulate specific aspects of information processing during cognitive control.

Identification of Breathing Patterns through EEG Signal Analysis Using Machine Learning

This study was to investigate the changes in brain function due to lack of oxygen (O₂) caused by mouth breathing, and to suggest a method to alleviate the side effects of mouth breathing on brain function through an additional O₂ supply. For this purpose, we classified the breathing patterns according to EEG signals using a machine learning technique and proposed a method to reduce the side effects of mouth breathing on brain function. Twenty subjects participated in this study, and each subject performed three different breathings: nose and mouth breathing and mouth breathing with O₂ supply during a working memory task. The results showed that nose breathing guarantees normal O₂ supply to the brain, but mouth breathing interrupts the O₂ supply to the brain. Therefore, this comparative study of EEG signals using machine learning showed that one of the most important elements distinguishing the effects of mouth and nose breathing on brain function was the difference in O₂ supply. These findings have important implications for the workplace environment, suggesting that special care is required for employees who work long hours in confined spaces such as public transport, and that a sufficient O₂ supply is needed in the workplace for working efficiency.

Temporally-precise disruption of prefrontal cortex informed by the timing of beta bursts impairs human action-stopping

Human action-stopping is thought to rely on a prefronto-basal ganglia-thalamocortical network, with right inferior frontal cortex (rIFC) posited to play a critical role in the early stage of implementation. Here we sought causal evidence for this idea in experiments involving healthy human participants. We first show that action-stopping is preceded by bursts of electroencephalographic activity in the beta band over prefrontal electrodes, putatively rIFC, and that the timing of these bursts correlates with the latency of stopping at a single-trial level: earlier bursts are associated with faster stopping. From this we reasoned that the integrity of rIFC at the time of beta bursts might be critical to successful stopping. We then used fMRI-guided transcranial magnetic stimulation (TMS) to disrupt rIFC at the approximate time of beta bursting. Stimulation prolonged stopping latencies and, moreover, the prolongation was most pronounced in individuals for whom the pulse appeared closer to the presumed time of beta bursting. These results help validate a model of the neural architecture and temporal dynamics of action-stopping. They also highlight the usefulness of prefrontal beta bursts to index an apparently important sub-process of stopping, the timing of which might help explain within- and between-individual variation in impulse control.