Riemannian geometry for EEG-based brain-computer interfaces; a primer and a review

Enhancing motor imagery EEG classification with a Riemannian geometry-based spatial filtering (RSF) method

Motor imagery (MI) refers to the mental simulation of movements without physical execution, and it can be captured using electroencephalography (EEG). This area has garnered significant research interest due to its substantial potential in brain-computer interface (BCI) applications, especially for individuals with physical disabilities. However, accurate classification of MI EEG signals remains a major challenge due to their non-stationary nature, low signal-to-noise ratio, and sensitivity to both external and physiological noise. Traditional classification methods, such as common spatial pattern (CSP), often assume that the data is stationary and Gaussian, which limits their applicability in real-world scenarios where these assumptions do not hold. These challenges highlight the need for more robust methods to improve classification accuracy in MI-BCI systems. To address these issues, this study introduces a Riemannian geometry-based spatial filtering (RSF) method that projects EEG signals into a lower-dimensional subspace, maximizing the Riemannian distance between covariance matrices from different classes. By leveraging the inherent geometric properties of EEG data, RSF enhances the discriminative power of the features while maintaining robustness against noise. The performance of RSF was evaluated in combination with ten commonly used MI decoding algorithms, including CSP with linear discriminant analysis (CSP-LDA), Filter Bank CSP (FBCSP), Minimum Distance to Riemannian Mean (MDM), Tangent Space Mapping (TSM), EEGNet, ShallowConvNet (sCNN), DeepConvNet (dCNN), FBCNet, Graph-CSPNet, and LMDA-Net, using six publicly available MI-BCI datasets. The results demonstrate that RSF significantly improves classification accuracy and reduces computational time, particularly for deep learning models with high computational complexity. These findings underscore the potential of RSF as an effective spatial filtering approach for MI EEG classification, providing new insights and opportunities for the development of robust MI-BCI systems. The code for this research is available at https://github.com/PLC-TJU/RSF.

Longitudinal voice monitoring in a decentralized Bring Your Own Device trial for respiratory illness detection

The Acute Respiratory Illness Surveillance (AcRIS) Study was a low-interventional trial that examined voice changes with respiratory illnesses. This longitudinal trial was the first of its kind, conducted in a fully decentralized manner via a Bring Your Own Device mobile application. The app enabled social-media-based recruitment, remote consent, at-home sample collection, and daily remote voice and symptom capture in real-world settings. From April 2021 to April 2022, the trial enrolled 9151 participants, followed for up to eight weeks. Despite mild symptoms experienced by reverse transcription polymerase chain reaction (RT-PCR) positive participants, two machine learning algorithms developed to screen respiratory illnesses reached the pre-specified success criteria. Algorithm testing on independent cohorts demonstrated that the algorithm's sensitivity increased as symptoms increased, while specificity remained consistent. Study findings suggest voice features can identify individuals with viral respiratory illnesses and provide valuable insights into fully decentralized clinical trials design, operation, and adoption (study registered at ClinicalTrials.gov (NCT04748445) on 5 February 2021).

The Riemannian Means Field Classifier for EEG-Based BCI Data

: A substantial amount of research has demonstrated the robustness and accuracy of the Riemannian minimum distance to mean (MDM) classifier for all kinds of EEG-based brain-computer interfaces (BCIs). This classifier is simple, fully deterministic, robust to noise, computationally efficient, and prone to transfer learning. Its training is very simple, requiring just the computation of a geometric mean of a symmetric positive-definite (SPD) matrix per class. We propose an improvement of the MDM involving a number of power means of SPD matrices instead of the sole geometric mean. By the analysis of 20 public databases, 10 for the motor-imagery BCI paradigm and 10 for the P300 BCI paradigm, comprising 587 individuals in total, we show that the proposed classifier clearly outperforms the MDM, approaching the state-of-the art in terms of performance while retaining the simplicity and the deterministic behavior. In order to promote reproducible research, our code will be released as open source.

Exploring the neuromagnetic signatures of cognitive decline from mild cognitive impairment to Alzheimer's disease dementia

BACKGROUND

Developing non-invasive and affordable biomarkers to detect Alzheimer's disease (AD) at a prodromal stage is essential, particularly in the context of new disease-modifying therapies. Mild cognitive impairment (MCI) is a critical stage preceding dementia, but not all patients with MCI will progress to AD. This study explores the potential of magnetoencephalography (MEG) to predict cognitive decline from MCI to AD dementia.

METHODS

We analysed resting-state MEG data from the BioFIND dataset including 117 patients with MCI among whom 64 developed AD dementia (AD progression), while 53 remained cognitively stable (stable MCI), using spectral analysis. Logistic regression models estimated the additive explanation of selected clinical, MEG, and MRI variables for AD progression risk. We then built a high-dimensional classification model to combine all modalities and variables of interest.

FINDINGS

MEG 16-38Hz spectral power, particularly over parieto-occipital magnetometers, was significantly reduced in the AD progression group. In logistic regression models, decreased MEG 16-38Hz spectral power and reduced hippocampal volume/total grey matter ratio on MRI were independently linked to higher AD progression risk. The data-driven classification model confirmed, among other factors, the complementary information of MEG covariance (AUC = 0.74, SD = 0.13) and MRI cortical volumes (AUC = 0.77, SD = 0.14) to predict AD progression. Combining all inputs led to markedly improved classification scores (AUC = 0.81, SD = 0.12).

INTERPRETATION

These findings highlight the potential of spectral power and covariance as robust non-invasive electrophysiological biomarkers to predict AD progression, complementing other diagnostic measures, including cognitive scores and MRI.

FUNDING

This work was supported by: Fondation pour la Recherche Médicale (grant FDM202106013579).

GREEN: A lightweight architecture using learnable wavelets and Riemannian geometry for biomarker exploration with EEG signals

Spectral analysis using wavelets is widely used for identifying biomarkers in EEG signals. Recently, Riemannian geometry has provided an effective mathematical framework for predicting biomedical outcomes from multichannel electroencephalography (EEG) recordings while showing concord with neuroscientific domain knowledge. However, these methods rely on handcrafted rules and sequential optimization. In contrast, deep learning (DL) offers end-to-end trainable models achieving state-of-the-art performance on various prediction tasks but lacks interpretability and interoperability with established neuroscience concepts. We introduce Gabor Riemann EEGNet (GREEN), a lightweight neural network that integrates wavelet transforms and Riemannian geometry for processing raw EEG data. Benchmarking on six prediction tasks across four datasets with over 5,000 participants, GREEN outperformed non-deep state-of-the-art models and performed favorably against large DL models while using orders-of-magnitude fewer parameters. Computational experiments showed that GREEN facilitates learning sparse representations without compromising performance. By integrating domain knowledge, GREEN combines a desirable complexity-performance trade-off with interpretable representations.

Tackling the possibility of extracting a brain digital fingerprint based on personal hobbies predilection

In an attempt to create a more familiar brain-machine interaction for biometric authentication applications, we investigated the efficiency of using the users' personal hobbies, interests, and memory collections. This approach creates a unique and pleasant experience that can be later utilized within an authentication protocol. This paper presents a new EEG dataset recorded while subjects watch images of popular hobbies, pictures with no point of interest and images with great personal significance. In addition, we propose several applications that can be tackled with our newly collected dataset. Namely, our study showcases 4 types of applications and we obtain state-of-the-art level results for all of them. The tackled tasks are: emotion classification, category classification, authorization process, and person identification. Our experiments show great potential for using EEG response to hobby visualization for people authentication. In our study, we show preliminary results for using predilection for personal hobbies, as measured by EEG, for identifying people. Also, we propose a novel authorization process paradigm using electroencephalograms. Code and dataset are available here.

A New Canonical Log-Euclidean Kernel for Symmetric Positive Definite Matrices for EEG Analysis (Oct 2024)

OBJECTIVE

Working with the Riemannian manifold of symmetric positive-definite (SPD) matrices has become popular in electroencephalography (EEG) analysis. Frequently selected for its speed property is the manifold geometry provided by the log-euclidean Riemannian metric. However, the kernels used in the log-euclidean framework are not canonically based on the underlying geometry. Therefore, we introduce a new canonical log-euclidean (CLE) kernel.

METHODS

We used the log-euclidean metric tensor on the SPD manifold to derive the CLE kernel. We compared it with existing kernels, namely the affine-invariant, log-euclidean, and Gaussian log-euclidean kernel. For comparison, we tested the kernels on two paradigms: classification and dimensionality reduction. Each paradigm was evaluated on five open-access brain-computer interface datasets with motor-imagery tasks across multiple sessions. Performance was measured as balanced classification accuracy using a leave-one-session-out cross-validation. Dimensionality reduction performance was measured using AUClogRNX.

RESULTS

The CLE kernel itself is simple and easily turned into code, which is provided in addition to all the analytical solutions to relevant equations in the log-euclidean framework. The CLE kernel significantly outperformed existing log-euclidean kernels in classification tasks and was several times faster than the affine-invariant kernel for most datasets.

CONCLUSION

We found that adhering to the geometrical structure significantly improves the accuracy over two commonly used log-euclidean kernels while keeping the speed advantages of the log-euclidean framework.

SIGNIFICANCE

The CLE provides a good choice as a kernel in time-critical applications and fills a gap in the kernel methods of the log-euclidean framework.

Evaluating robotic actions: spatiotemporal brain dynamics of performance assessment in robot-assisted laparoscopic training

Introduction

Enhancing medical robot training traditionally relies on explicit feedback from physicians to identify optimal and suboptimal robotic actions during surgery. Passive brain-computer interfaces (BCIs) offer an emerging alternative by enabling implicit brain-based performance evaluations. However, effectively decoding these evaluations of robot performance requires a comprehensive understanding of the spatiotemporal brain dynamics identifying optimal and suboptimal robot actions within realistic settings.

Methods

We conducted an electroencephalographic study with 16 participants who mentally assessed the quality of robotic actions while observing simulated robot-assisted laparoscopic surgery scenarios designed to approximate real-world conditions. We aimed to identify key spatiotemporal dynamics using the surface Laplacian technique and two complementary data-driven methods: a mass-univariate permutation-based clustering and multivariate pattern analysis (MVPA)-based temporal decoding. A second goal was to identify the optimal time interval of evoked brain signatures for single-trial classification.

Results

Our analyses revealed three distinct spatiotemporal brain dynamics differentiating the quality assessment of optimal vs. suboptimal robotic actions during video-based laparoscopic training observations. Specifically, an enhanced left fronto-temporal current source, consistent with P300, LPP, and P600 components, indicated heightened attentional allocation and sustained evaluation processes during suboptimal robot actions. Additionally, amplified current sinks in right frontal and mid-occipito-parietal regions suggested prediction-based processing and conflict detection, consistent with the oERN and interaction-based ERN/N400. Both mass-univariate clustering and MVPA provided convergent evidence supporting these neural distinctions.

Discussion

The identified neural signatures propose that suboptimal robotic actions elicit enhanced, sustained brain dynamics linked to continuous attention allocation, action monitoring, conflict detection, and ongoing evaluative processing. The findings highlight the importance of prioritizing late evaluative brain signatures in BCIs to classify robotic actions reliably. These insights have significant implications for advancing machine-learning-based training paradigms.

Hybrid BCI for Meal-Assist Robot Using Dry-Type EEG and Pupillary Light Reflex

Brain-computer interface (BCI)-based assistive technologies enable intuitive and efficient user interaction, significantly enhancing the independence and quality of life of elderly and disabled individuals. Although existing wet EEG-based systems report high accuracy, they suffer from limited practicality. This study presents a hybrid BCI system combining dry-type EEG-based flash visual-evoked potentials (FVEP) and pupillary light reflex (PLR) designed to control an LED-based meal-assist robot. The hybrid system integrates dry-type EEG and eyewear-type infrared cameras, addressing the preparation challenges of wet electrodes, while maintaining practical usability and high classification performance. Offline experiments demonstrated an average accuracy of 88.59% and an information transfer rate (ITR) of 18.23 bit/min across the four target classifications. Real-time implementation uses PLR triggers to initiate the meal cycle and EMG triggers to detect chewing, indicating the completion of the cycle. These features allow intuitive and efficient operation of the meal-assist robot. This study advances the BCI-based assistive technologies by introducing a hybrid system optimized for real-world applications. The successful integration of the FVEP and PLR in a meal-assisted robot demonstrates the potential for robust and user-friendly solutions that empower the users with autonomy and dignity in their daily activities.

Latent alignment in deep learning models for EEG decoding

Objective. Brain-computer interfaces (BCIs) face a significant challenge due to variability in electroencephalography (EEG) signals across individuals. While recent approaches have focused on standardizing input signal distributions, we propose that aligning distributions in the deep learning model's feature space is more effective for classification.Approach. We introduce the Latent Alignment method, which won the Benchmarks for EEG Transfer Learning competition. This method can be formulated as a deep set architecture applied to trials from a given subject, introducing deep sets to EEG decoding for the first time. We compare Latent Alignment to recent statistical domain adaptation techniques, carefully considering class-discriminative artifacts and the impact of class distributions on classification performance.Main results. Our experiments across motor imagery, sleep stage classification, and P300 event-related potential tasks validate Latent Alignment's effectiveness. We identify a trade-off between improved classification accuracy when alignment is performed at later modeling stages and increased susceptibility to class imbalance in the trial set used for statistical computation.Significance. Latent Alignment offers consistent improvements to subject-independent deep learning models for EEG decoding when relevant practical considerations are addressed. This work advances our understanding of statistical alignment techniques in EEG decoding and provides insights for their effective implementation in real-world BCI applications, potentially facilitating broader use of BCIs in healthcare, assistive technologies, and beyond. The model code is available athttps://github.com/StylianosBakas/LatentAlignment.

Novel machine learning-driven comparative analysis of CSP, STFT, and CSP-STFT fusion for EEG data classification across multiple meditation and non-meditation sessions in BCI pipeline

This study focuses on classifying multiple sessions of loving kindness meditation (LKM) and non-meditation electroencephalography (EEG) data. This novel study focuses on using multiple sessions of EEG data from a single individual to train a machine learning pipeline, and then using a new session data from the same individual for the classification. Here, two meditation techniques, LKM-Self and LKM-Others were compared with non-meditation EEG data for 12 participants. Among many tested, three BCI pipelines we built produced promising results, successfully detecting features in meditation/ non-meditation EEG data. While testing different feature extraction algorithms, a common neural network structure was used as the classification algorithm to compare the performance of the feature extraction algorithms. For two of those pipelines, Common Spatial Patterns (CSP) and Short Time Fourier Transform (STFT) were successfully used as feature extraction algorithms where both these algorithms are significantly new for meditation EEG. As a novel concept, the third BCI pipeline used a feature extraction algorithm that fused the features of CSP and STFT, achieving the highest classification accuracies among all tested pipelines. Analyses were conducted using EEG data of 3, 4 or 5 sessions, totaling 3960 tests on the entire dataset. At the end of the study, when considering all the tests, the overall classification accuracy using SCP alone was 67.1%, and it was 67.8% for STFT alone. The algorithm combining the features of CSP and STFT achieved an overall classification accuracy of 72.9% which is more than 5% higher than the other two pipelines. At the same time, the highest mean classification accuracy for the 12 participants was achieved using the pipeline with the combination of CSP STFT algorithm, reaching 75.5% for LKM-Self/ non-meditation for the case of 5 sessions of data. Additionally, the highest individual classification accuracy of 88.9% was obtained by the participant no. 14. Furthermore, the results showed that the classification accuracies for all three pipelines increased with the number of training sessions increased from 2 to 3 and then to 4. The study was successful in classifying a new session of EEG meditation/ non-meditation data after training machine learning algorithms using a different set of session data, and this achievement will be beneficial in the development of algorithms that support meditation.

Repairbads: An automatic and adaptive method to repair bad channels and segments for OPM-MEG

The optically pumped magnetometer (OPM) based magnetoencephalography (MEG) system offers advantages such as flexible layout and wearability. However, the position instability or jitter of OPM sensors can result in bad channels and segments, which significantly impede subsequent preprocessing and analysis. Most common methods directly reject or interpolate to repair these bad channels and segments. Direct rejection leads to data loss, and when the number of sensors is limited, interpolation using neighboring sensors can cause significant signal distortion and cannot repair bad segments present in all channels. Therefore, most existing methods are unsuitable for OPM-MEG systems with fewer channels. We introduce an automatic bad segments and bad channels repair method for OPM-MEG, called Repairbads. This method aims to repair all bad data and reduce signal distortion, especially capable of automatically repairing bad segments present in all channels simultaneously. Repairbads employs Riemannian Potato combined with joint decorrelation to project out artifact components, achieving automatic bad segment repair. Then, an adaptive algorithm is used to segment the signal into relatively stable noise data chunks, and the source-estimate-utilizing noise-discarding algorithm is applied to each chunk to achieve automatic bad channel repair. We compared the performance of Repairbads with the Autoreject method on both simulated and real auditory evoked data, using five evaluation metrics for quantitative assessment. The results demonstrate that Repairbads consistently outperforms across all five metrics. In both simulated and real OPM-MEG data, Repairbads shows better performance than current state-of-the-art methods, reliably repairing bad data with minimal distortion. The automation of this method significantly reduces the burden of manual inspection, promoting the automated processing and clinical application of OPM-MEG.

Who is WithMe? EEG features for attention in a visual task, with auditory and rhythmic support

Introduction

The study of attention has been pivotal in advancing our comprehension of cognition. The goal of this study is to investigate which EEG data representations or features are most closely linked to attention, and to what extent they can handle the cross-subject variability.

Methods

We explore the features obtained from the univariate time series from a single EEG channel, such as time domain features and recurrence plots, as well as representations obtained directly from the multivariate time series, such as global field power or functional brain networks. To address the cross-subject variability in EEG data, we also investigate persistent homology features that are robust to different types of noise. The performance of the different EEG representations is evaluated with the Support Vector Machine (SVM) accuracy on the WithMe data derived from a modified digit span experiment, and is benchmarked against baseline EEG-specific models, including a deep learning architecture known for effectively learning task-specific features.

Results

The raw EEG time series outperform each of the considered data representations, but can fall short in comparison with the black-box deep learning approach that learns the best features.

Discussion

The findings are limited to the WithMe experimental paradigm, highlighting the need for further studies on diverse tasks to provide a more comprehensive understanding of their utility in the analysis of EEG data.

Multimodal consumer choice prediction using EEG signals and eye tracking

Marketing plays a vital role in the success of a business, driving customer engagement, brand recognition, and revenue growth. Neuromarketing adds depth to this by employing insights into consumer behavior through brain activity and emotional responses to create more effective marketing strategies. Electroencephalogram (EEG) has typically been utilized by researchers for neuromarketing, whereas Eye Tracking (ET) has remained unexplored. To address this gap, we propose a novel multimodal approach to predict consumer choices by integrating EEG and ET data. Noise from EEG signals is mitigated using a bandpass filter, Artifact Subspace Reconstruction (ASR), and Fast Orthogonal Regression for Classification and Estimation (FORCE). Class imbalance is handled by employing the Synthetic Minority Over-sampling Technique (SMOTE). Handcrafted features, including statistical and wavelet features, and automated features from Convolutional Neural Network and Long Short-Term Memory (CNN-LSTM), have been extracted and concatenated to generate a feature space representation. For ET data, preprocessing involved interpolation, gaze plots, and SMOTE, followed by feature extraction using LeNet-5 and handcrafted features like fixations and saccades. Multimodal feature space representation was generated by performing feature-level fusion for EEG and ET, which was later fed into a meta-learner-based ensemble classifier with three base classifiers, including Random Forest, Extended Gradient Boosting, and Gradient Boosting, and Random Forest as the meta-classifier, to perform classification between buy vs. not buy. The performance of the proposed approach is evaluated using a variety of performance metrics, including accuracy, precision, recall, and F1 score. Our model demonstrated superior performance compared to competitors by achieving 84.01% accuracy in predicting consumer choices and 83% precision in identifying positive consumer preferences.

Novel AIRTrode-based wearable electrode supports long-term, online brain-computer interface operations

Objective.Non-invasive electroencephalograms (EEG)-based brain-computer interfaces (BCIs) play a crucial role in a diverse range of applications, including motor rehabilitation, assistive and communication technologies, holding potential promise to benefit users across various clinical spectrums. Effective integration of these applications into daily life requires systems that provide stable and reliable BCI control for extended periods. Our prior research introduced the AIRTrode, a self-adhesive (A), injectable (I), and room-temperature (RT) spontaneously-crosslinked hydrogel electrode (AIRTrode). The AIRTrode has shown lower skin-contact impedance and greater stability than dry electrodes and, unlike wet gel electrodes, does not dry out after just a few hours, enhancing its suitability for long-term application. This study aims to demonstrate the efficacy of AIRTrodes in facilitating reliable, stable and long-term online EEG-based BCI operations.Approach.In this study, four healthy participants utilized AIRTrodes in two BCI control tasks-continuous and discrete-across two sessions separated by six hours. Throughout this duration, the AIRTrodes remained attached to the participants' heads. In the continuous task, participants controlled the BCI through decoding of upper-limb motor imagery (MI). In the discrete task, the control was based on decoding of error-related potentials (ErrPs).Main Results.Using AIRTrodes, participants demonstrated consistently reliable online BCI performance across both sessions and tasks. The physiological signals captured during MI and ErrPs tasks were valid and remained stable over sessions. Lastly, both the BCI performances and physiological signals captured were comparable with those from freshly applied, research-grade wet gel electrodes, the latter requiring inconvenient re-application at the start of the second session.Significance.AIRTrodes show great potential promise for integrating non-invasive BCIs into everyday settings due to their ability to support consistent BCI performances over extended periods. This technology could significantly enhance the usability of BCIs in real-world applications, facilitating continuous, all-day functionality that was previously challenging with existing electrode technologies.

A Task-Related EEG Microstate Clustering Algorithm Based on Spatial Patterns, Riemannian Distance, and a Deep Autoencoder

BACKGROUND

The segmentation of electroencephalography (EEG) signals into a limited number of microstates is of significant importance in the field of cognitive neuroscience. Currently, the microstate analysis algorithm based on global field power has demonstrated its efficacy in clustering resting-state EEG. The task-related EEG was extensively analyzed in the field of brain-computer interfaces (BCIs); however, its primary objective is classification rather than segmentation.

METHODS

We propose an innovative algorithm for analyzing task-related EEG microstates based on spatial patterns, Riemannian distance, and a modified deep autoencoder. The objective of this algorithm is to achieve unsupervised segmentation and clustering of task-related EEG signals.

RESULTS

The proposed algorithm was validated through experiments conducted on simulated EEG data and two publicly available cognitive task datasets. The evaluation results and statistical tests demonstrate its robustness and efficiency in clustering task-related EEG microstates.

CONCLUSIONS

The proposed unsupervised algorithm can autonomously discretize EEG signals into a finite number of microstates, thereby facilitating investigations into the temporal structures underlying cognitive processes.

Improving subject transfer in EEG classification with divergence estimation

Objective. Classification models for electroencephalogram (EEG) data show a large decrease in performance when evaluated on unseen test subjects. We improve performance using new regularization techniques during model training.Approach. We propose several graphical models to describe an EEG classification task. From each model, we identify statistical relationships that should hold true in an idealized training scenario (with infinite data and a globally-optimal model) but that may not hold in practice. We design regularization penalties to enforce these relationships in two stages. First, we identify suitable proxy quantities (divergences such as Mutual Information and Wasserstein-1) that can be used to measure statistical independence and dependence relationships. Second, we provide algorithms to efficiently estimate these quantities during training using secondary neural network models.Main results. We conduct extensive computational experiments using a large benchmark EEG dataset, comparing our proposed techniques with a baseline method that uses an adversarial classifier. We first show the performance of each method across a wide range of hyperparameters, demonstrating that each method can be easily tuned to yield significant benefits over an unregularized model. We show that, using ideal hyperparameters for all methods, our first technique gives significantly better performance than the baseline regularization technique. We also show that, across hyperparameters, our second technique gives significantly more stable performance than the baseline. The proposed methods require only a small computational cost at training time that is equivalent to the cost of the baseline.Significance. The high variability in signal distribution between subjects means that typical approaches to EEG signal modeling often require time-intensive calibration for each user, and even re-calibration before every use. By improving the performance of population models in the most stringent case of zero-shot subject transfer, we may help reduce or eliminate the need for model calibration.

Graph Neural Networks on SPD Manifolds for Motor Imagery Classification: A Perspective From the Time-Frequency Analysis

The motor imagery (MI) classification has been a prominent research topic in brain-computer interfaces (BCIs) based on electroencephalography (EEG). Over the past few decades, the performance of MI-EEG classifiers has seen gradual enhancement. In this study, we amplify the geometric deep-learning-based MI-EEG classifiers from the perspective of time-frequency analysis, introducing a new architecture called Graph-CSPNet. We refer to this category of classifiers as Geometric Classifiers, highlighting their foundation in differential geometry stemming from EEG spatial covariance matrices. Graph-CSPNet utilizes novel manifold-valued graph convolutional techniques to capture the EEG features in the time-frequency domain, offering heightened flexibility in signal segmentation for capturing localized fluctuations. To evaluate the effectiveness of Graph-CSPNet, we employ five commonly used publicly available MI-EEG datasets, achieving near-optimal classification accuracies in nine out of 11 scenarios. The Python repository can be found at https://github.com/GeometricBCI/Tensor-CSPNet-and-Graph-CSPNet.

Coherence-based channel selection and Riemannian geometry features for magnetoencephalography decoding

Magnetoencephalography (MEG) records the extremely weak magnetic fields on the surface of the scalp through highly sensitive sensors. Multi-channel MEG data provide higher spatial and temporal resolution when measuring brain activities, and can be applied for brain-computer interfaces as well. However, a large number of channels leads to high computational complexity and can potentially impact decoding accuracy. To improve the accuracy of MEG decoding, this paper proposes a new coherence-based channel selection method that effectively identifies task-relevant channels, reducing the presence of noisy and redundant information. Riemannian geometry is then used to extract effective features from selected channels of MEG data. Finally, MEG decoding is achieved by training a support vector machine classifier with the Radial Basis Function kernel. Experiments were conducted on two public MEG datasets to validate the effectiveness of the proposed method. The results from Dataset 1 show that Riemannian geometry achieves higher classification accuracy (compared to common spatial patterns and power spectral density) in the single-subject visual decoding task. Moreover, coherence-based channel selection significantly (P = 0.0002) outperforms the use of all channels. Moving on to Dataset 2, the results reveal that coherence-based channel selection is also significantly (P <0.0001) superior to all channels and channels around C3 and C4 in cross-session mental imagery decoding tasks. Additionally, the proposed method outperforms state-of-the-art methods in motor imagery tasks.

Refining sleep staging accuracy: transfer learning coupled with scorability models

STUDY OBJECTIVES

This study aimed to (1) improve sleep staging accuracy through transfer learning (TL), to achieve or exceed human inter-expert agreement and (2) introduce a scorability model to assess the quality and trustworthiness of automated sleep staging.

METHODS

A deep neural network (base model) was trained on a large multi-site polysomnography (PSG) dataset from the United States. TL was used to calibrate the model to a reduced montage and limited samples from the Korean Genome and Epidemiology Study (KoGES) dataset. Model performance was compared to inter-expert reliability among three human experts. A scorability assessment was developed to predict the agreement between the model and human experts.

RESULTS

Initial sleep staging by the base model showed lower agreement with experts (κ = 0.55) compared to the inter-expert agreement (κ = 0.62). Calibration with 324 randomly sampled training cases matched expert agreement levels. Further targeted sampling improved performance, with models exceeding inter-expert agreement (κ = 0.70). The scorability assessment, combining biosignal quality and model confidence features, predicted model-expert agreement moderately well (R² = 0.42). Recordings with higher scorability scores demonstrated greater model-expert agreement than inter-expert agreement. Even with lower scorability scores, model performance was comparable to inter-expert agreement.

CONCLUSIONS

Fine-tuning a pretrained neural network through targeted TL significantly enhances sleep staging performance for an atypical montage, achieving and surpassing human expert agreement levels. The introduction of a scorability assessment provides a robust measure of reliability, ensuring quality control and enhancing the practical application of the system before deployment. This approach marks an important advancement in automated sleep analysis, demonstrating the potential for AI to exceed human performance in clinical settings.

Global motor dynamics - Invariant neural representations of motor behavior in distributed brain-wide recordings

Objective.Motor-related neural activity is more widespread than previously thought, as pervasive brain-wide neural correlates of motor behavior have been reported in various animal species. Brain-wide movement-related neural activity have been observed in individual brain areas in humans as well, but it is unknown to what extent global patterns exist.Approach.Here, we use a decoding approach to capture and characterize brain-wide neural correlates of movement. We recorded invasive electrophysiological data from stereotactic electroencephalographic electrodes implanted in eight epilepsy patients who performed both an executed and imagined grasping task. Combined, these electrodes cover the whole brain, including deeper structures such as the hippocampus, insula and basal ganglia. We extract a low-dimensional representation and classify movement from rest trials using a Riemannian decoder.Main results.We reveal global neural dynamics that are predictive across tasks and participants. Using an ablation analysis, we demonstrate that these dynamics remain remarkably stable under loss of information. Similarly, the dynamics remain stable across participants, as we were able to predict movement across participants using transfer learning.Significance.Our results show that decodable global motor-related neural dynamics exist within a low-dimensional space. The dynamics are predictive of movement, nearly brain-wide and present in all our participants. The results broaden the scope to brain-wide investigations, and may allow combining datasets of multiple participants with varying electrode locations or calibrationless neural decoder.

Development of Machine Learning Algorithms Using EEG Data to Detect the Presence of Chronic Pain

Chronic pain impacts more than one in five adults in the United States (US) and the costs associated with the condition amount to hundreds of billions of dollars annually. Despite the tremendous impact of chronic pain in the US and worldwide, the standard of care for diagnosis depends on subjective self-reporting of pain state, with no effective objective assessment procedure available. This study investigated the application of signal processing and machine learning to electroencephalography (EEG) data for the development of classification algorithms capable of differentiating subjects in pain from pain free subjects. In this study, nineteen (19) channels of EEG data were obtained from subjects in an eyes closed resting state, and ultimately data from 186 participants were used for algorithm development, including 35 healthy controls and 151 chronic pain patients. Signal processing was applied to identify noise free segments of EEG data and 6375 quantitative EEG (qEEG) measures were calculated for each subject. Various machine learning methodologies were applied to the data, with Elastic Net chosen as the optimal methodology. The final classifier developed using Elastic Net contained 34 qEEG features with non-zero weights. The classifier was able to differentiate pain versus no pain subjects with an accuracy of 79.6%, sensitivity of 82.2%, and specificity of 66.7%. The features used in the classifier were evaluated and found to align well with contemporary literature regarding changes in neurological characteristics associated with chronic pain.

Predicting cognitive load with EEG using Riemannian geometry-based features

Objective. We show that electroencephalography (EEG)-based cognitive load (CL) prediction using Riemannian geometry features outperforms existing models. The performance is estimated using Riemannian Procrustes Analysis (RPA) with a test set of subjects unseen during training.Approach. Performance is evaluated by using the Minimum Distance to Riemannian Mean model trained on CL classification. The baseline performance is established using spatial covariance matrices of the signal as features. Various novel features are explored and analyzed in depth, including spatial covariance and correlation matrices computed on the EEG signal and its first-order derivative. Furthermore, each RPA step effect on the performance is investigated, and the generalization performance of RPA is compared against a few different generalization methods.Main results. Performances are greatly improved by using the spatial covariance matrix of the first-order derivative of the signal as features. Furthermore, this work highlights both the importance and efficiency of RPA for CL prediction: it achieves good generalizability with little amounts of calibration data and largely outperforms all the comparison methods.Significance. CL prediction using RPA for generalizability across subjects is an approach worth exploring further, especially for real-world applications where calibration time is limited. Furthermore, the feature exploration uncovers new, promising features that can be used and further experimented within any Riemannian geometry setting.

ℛSCZ: A Riemannian schizophrenia diagnosis framework based on the multiplexity of EEG-based dynamic functional connectivity patterns

Abnormal electrophysiological (EEG) activity has been largely reported in schizophrenia (SCZ). In the last decade, research has focused to the automatic diagnosis of SCZ via the investigation of an EEG aberrant activity and connectivity linked to this mental disorder. These studies followed various preprocessing steps of EEG activity focusing on frequency-dependent functional connectivity brain network (FCBN) construction disregarding the topological dependency among edges. FCBN belongs to a family of symmetric positive definite (SPD) matrices forming the Riemannian manifold. Due to its unique geometric properties, the whole analysis of FCBN can be performed on the Riemannian geometry of the SPD space. The advantage of the analysis of FCBN on the SPD space is that it takes into account all the pairwise interdependencies as a whole. However, only a few studies have adopted a FCBN analysis on the SPD manifold, while no study exists on the analysis of dynamic FCBN (dFCBN) tailored to SCZ. In the present study, I analyzed two open EEG-SCZ datasets under a Riemannian geometry of SPD matrices for the dFCBN analysis proposing also a multiplexity index that quantifies the associations of multi-frequency brainwave patterns. I adopted a machine learning procedure employing a leave-one-subject-out cross-validation (LOSO-CV) using snapshots of dFCBN from (N-1) subjects to train a battery of classifiers. Each classifier operated in the inter-subject dFCBN distances of sample covariance matrices (SCMs) following a rhythm-dependent decision and a multiplex-dependent one. The proposed ℛSCZ decoder supported both the Riemannian geometry of SPD and the multiplexity index DC reaching an absolute accuracy (100 %) in both datasets in the virtual default mode network (DMN) source space.

Classification of BCI-EEG Based on the Augmented Covariance Matrix

OBJECTIVE

Electroencephalography signals are recorded as multidimensional datasets. We propose a new framework based on the augmented covariance that stems from an autoregressive model to improve motor imagery classification.

METHODS

From the autoregressive model can be derived the Yule-Walker equations, which show the emergence of a symmetric positive definite matrix: the augmented covariance matrix. The state-of the art for classifying covariance matrices is based on Riemannian Geometry. A fairly natural idea is therefore to apply this Riemannian Geometry based approach to these augmented covariance matrices. The methodology for creating the augmented covariance matrix shows a natural connection with the delay embedding theorem proposed by Takens for dynamical systems. Such an embedding method is based on the knowledge of two parameters: the delay and the embedding dimension, respectively related to the lag and the order of the autoregressive model. This approach provides new methods to compute the hyper-parameters in addition to standard grid search.

RESULTS

The augmented covariance matrix performed ACMs better than any state-of-the-art methods. We will test our approach on several datasets and several subjects using the MOABB framework, using both within-session and cross-session evaluation.

CONCLUSION

The improvement in results is due to the fact that the augmented covariance matrix incorporates not only spatial but also temporal information. As such, it contains information on the nonlinear components of the signal through the embedding procedure, which allows the leveraging of dynamical systems algorithms.

SIGNIFICANCE

These results extend the concepts and the results of the Riemannian distance based classification algorithm.

Machine learning of brain-specific biomarkers from EEG

BACKGROUND

Electroencephalography (EEG) has a long history as a clinical tool to study brain function, and its potential to derive biomarkers for various applications is far from exhausted. Machine learning (ML) can guide future innovation by harnessing the wealth of complex EEG signals to isolate relevant brain activity. Yet, ML studies in EEG tend to ignore physiological artefacts, which may cause problems for deriving biomarkers specific to the central nervous system (CNS).

METHODS

We present a framework for conceptualising machine learning from CNS versus peripheral signals measured with EEG. A signal representation based on Morlet wavelets allowed us to define traditional brain activity features (e.g. log power) and alternative inputs used by state-of-the-art ML approaches based on covariance matrices. Using more than 2600 EEG recordings from large public databases (TUAB, TDBRAIN), we studied the impact of peripheral signals and artefact removal techniques on ML models in age and sex prediction analyses.

FINDINGS

Across benchmarks, basic artefact rejection improved model performance, whereas further removal of peripheral signals using ICA decreased performance. Our analyses revealed that peripheral signals enable age and sex prediction. However, they explained only a fraction of the performance provided by brain signals.

INTERPRETATION

We show that brain signals and body signals, both present in the EEG, allow for prediction of personal characteristics. While these results may depend on specific applications, our work suggests that great care is needed to separate these signals when the goal is to develop CNS-specific biomarkers using ML.

FUNDING

All authors have been working for F. Hoffmann-La Roche Ltd.

Multi-Resolution Wavelet Fractal Analysis and Subtask Training for Enhancing Few-Shot Noisy Brainwave Recognition

The integration of healthcare monitoring with Internet of Things (IoT) networks radically transforms the management and monitoring of human well-being. Portable and lightweight electroencephalography (EEG) systems with fewer electrodes have improved convenience and flexibility while retaining adequate accuracy. However, challenges emerge when dealing with real-time EEG data from IoT devices due to the presence of noisy samples, which impedes improvements in brainwave detection accuracy. Moreover, high inter-subject variability and substantial variability in EEG signals present difficulties for conventional data augmentation and subtask learning techniques, leading to poor generalizability. To address these issues, we present a novel framework for enhancing EEG-based recognition through multi-resolution data analysis, capturing features at different scales using wavelet fractals. The original data can be expanded many times after continuous wavelet transform (CWT) and recombination, alleviating insufficient training samples. In the transfer stage of deep learning (DL) models, we adopt a subtask learning approach to train the recognition model to generalize efficiently. This incorporates wavelets at various scales instead of exclusively considering average prediction performance across scales and paradigms. Through extensive experiments, we demonstrate that our proposed DL-based method excels at extracting features from small-scale and noisy EEG data. This significantly improves healthcare monitoring performance by mitigating the impact of noise introduced by the external environment.

A systematic evaluation of Euclidean alignment with deep learning for EEG decoding

Objective:Electroencephalography signals are frequently used for various Brain-Computer interface (BCI) tasks. While deep learning (DL) techniques have shown promising results, they are hindered by the substantial data requirements. By leveraging data from multiple subjects, transfer learning enables more effective training of DL models. A technique that is gaining popularity is Euclidean alignment (EA) due to its ease of use, low computational complexity, and compatibility with DL models. However, few studies evaluate its impact on the training performance of shared and individual DL models. In this work, we systematically evaluate the effect of EA combined with DL for decoding BCI signals.Approach:We used EA as a pre-processing step to train shared DL models with data from multiple subjects and evaluated their transferability to new subjects.Main results:Our experimental results show that it improves decoding in the target subject by 4.33% and decreases convergence time by more than 70%. We also trained individual models for each subject to use as a majority-voting ensemble classifier. In this scenario, using EA improved the 3-model ensemble accuracy by 3.71%. However, when compared to the shared model with EA, the ensemble accuracy was 3.62% lower.Significance:EA succeeds in the task of improving transfer learning performance with DL models and, could be used as a standard pre-processing technique.

A 3D ray traced biological neural network learning model

Training large neural networks on big datasets requires significant computational resources and time. Transfer learning reduces training time by pre-training a base model on one dataset and transferring the knowledge to a new model for another dataset. However, current choices of transfer learning algorithms are limited because the transferred models always have to adhere to the dimensions of the base model and can not easily modify the neural architecture to solve other datasets. On the other hand, biological neural networks (BNNs) are adept at rearranging themselves to tackle completely different problems using transfer learning. Taking advantage of BNNs, we design a dynamic neural network that is transferable to any other network architecture and can accommodate many datasets. Our approach uses raytracing to connect neurons in a three-dimensional space, allowing the network to grow into any shape or size. In the Alcala dataset, our transfer learning algorithm trains the fastest across changing environments and input sizes. In addition, we show that our algorithm also outperformance the state of the art in EEG dataset. In the future, this network may be considered for implementation on real biological neural networks to decrease power consumption.

An auditory brain-computer interface based on selective attention to multiple tone streams

In this study, we attempted to improve brain-computer interface (BCI) systems by means of auditory stream segregation in which alternately presented tones are perceived as sequences of various different tones (streams). A 3-class BCI using three tone sequences, which were perceived as three different tone streams, was investigated and evaluated. Each presented musical tone was generated by a software synthesizer. Eleven subjects took part in the experiment. Stimuli were presented to each user's right ear. Subjects were requested to attend to one of three streams and to count the number of target stimuli in the attended stream. In addition, 64-channel electroencephalogram (EEG) and two-channel electrooculogram (EOG) signals were recorded from participants with a sampling frequency of 1000 Hz. The measured EEG data were classified based on Riemannian geometry to detect the object of the subject's selective attention. P300 activity was elicited by the target stimuli in the segregated tone streams. In five out of eleven subjects, P300 activity was elicited only by the target stimuli included in the attended stream. In a 10-fold cross validation test, a classification accuracy over 80% for five subjects and over 75% for nine subjects was achieved. For subjects whose accuracy was lower than 75%, either the P300 was also elicited for nonattended streams or the amplitude of P300 was small. It was concluded that the number of selected BCI systems based on auditory stream segregation can be increased to three classes, and these classes can be detected by a single ear without the aid of any visual modality.

Advancing passive BCIs: a feasibility study of two temporal derivative features and effect size-based feature selection in continuous online EEG-based machine error detection

The emerging integration of Brain-Computer Interfaces (BCIs) in human-robot collaboration holds promise for dynamic adaptive interaction. The use of electroencephalogram (EEG)-measured error-related potentials (ErrPs) for online error detection in assistive devices offers a practical method for improving the reliability of such devices. However, continuous online error detection faces challenges such as developing efficient and lightweight classification techniques for quick predictions, reducing false alarms from artifacts, and dealing with the non-stationarity of EEG signals. Further research is essential to address the complexities of continuous classification in online sessions. With this study, we demonstrated a comprehensive approach for continuous online EEG-based machine error detection, which emerged as the winner of a competition at the 32nd International Joint Conference on Artificial Intelligence. The competition consisted of two stages: an offline stage for model development using pre-recorded, labeled EEG data, and an online stage 3 months after the offline stage, where these models were tested live on continuously streamed EEG data to detect errors in orthosis movements in real time. Our approach incorporates two temporal-derivative features with an effect size-based feature selection technique for model training, together with a lightweight noise filtering method for online sessions without recalibration of the model. The model trained in the offline stage not only resulted in a high average cross-validation accuracy of 89.9% across all participants, but also demonstrated remarkable performance during the online session 3 months after the initial data collection without further calibration, maintaining a low overall false alarm rate of 1.7% and swift response capabilities. Our research makes two significant contributions to the field. Firstly, it demonstrates the feasibility of integrating two temporal derivative features with an effect size-based feature selection strategy, particularly in online EEG-based BCIs. Secondly, our work introduces an innovative approach designed for continuous online error prediction, which includes a straightforward noise rejection technique to reduce false alarms. This study serves as a feasibility investigation into a methodology for seamless error detection that promises to transform practical applications in the domain of neuroadaptive technology and human-robot interaction.

EEG functional connectivity as a Riemannian mediator: An application to malnutrition and cognition

Mediation analysis assesses whether an exposure directly produces changes in cognitive behavior or is influenced by intermediate "mediators". Electroencephalographic (EEG) spectral measurements have been previously used as effective mediators representing diverse aspects of brain function. However, it has been necessary to collapse EEG measures onto a single scalar using standard mediation methods. In this article, we overcome this limitation and examine EEG frequency-resolved functional connectivity measures as a mediator using the full EEG cross-spectral tensor (CST). Since CST samples do not exist in Euclidean space but in the Riemannian manifold of positive-definite tensors, we transform the problem, allowing for the use of classic multivariate statistics. Toward this end, we map the data from the original manifold space to the Euclidean tangent space, eliminating redundant information to conform to a "compressed CST." The resulting object is a matrix with rows corresponding to frequencies and columns to cross spectra between channels. We have developed a novel matrix mediation approach that leverages a nuclear norm regularization to determine the matrix-valued regression parameters. Furthermore, we introduced a global test for the overall CST mediation and a test to determine specific channels and frequencies driving the mediation. We validated the method through simulations and applied it to our well-studied 50+-year Barbados Nutrition Study dataset by comparing EEGs collected in school-age children (5-11 years) who were malnourished in the first year of life with those of healthy classmate controls. We hypothesized that the CST mediates the effect of malnutrition on cognitive performance. We can now explicitly pinpoint the frequencies (delta, theta, alpha, and beta bands) and regions (frontal, central, and occipital) in which functional connectivity was altered in previously malnourished children, an improvement to prior studies. Understanding the specific networks impacted by a history of postnatal malnutrition could pave the way for developing more targeted and personalized therapeutic interventions. Our methods offer a versatile framework applicable to mediation studies encompassing matrix and Hermitian 3D tensor mediators alongside scalar exposures and outcomes, facilitating comprehensive analyses across diverse research domains.

Age-Related Characteristics of Resting-State Electroencephalographic Signals and the Corresponding Analytic Approaches: A Review

The study of the effects of aging on neural activity in the human brain has attracted considerable attention in neurophysiological, neuropsychiatric, and neurocognitive research, as it is directly linked to an understanding of the neural mechanisms underlying the disruption of the brain structures and functions that lead to age-related pathological disorders. Electroencephalographic (EEG) signals recorded during resting-state conditions have been widely used because of the significant advantage of non-invasive signal acquisition with higher temporal resolution. These advantages include the capability of a variety of linear and nonlinear signal analyses and state-of-the-art machine-learning and deep-learning techniques. Advances in artificial intelligence (AI) can not only reveal the neural mechanisms underlying aging but also enable the assessment of brain age reliably by means of the age-related characteristics of EEG signals. This paper reviews the literature on the age-related features, available analytic methods, large-scale resting-state EEG databases, interpretations of the resulting findings, and recent advances in age-related AI models.

Pathway Activation Analysis for Pan-Cancer Personalized Characterization Based on Riemannian Manifold

The pathogenesis of carcinoma is believed to come from the combined effect of polygenic variation, and the initiation and progression of malignant tumors are closely related to the dysregulation of biological pathways. Quantifying the alteration in pathway activation and identifying coordinated patterns of pathway dysfunction are the imperative part of understanding the malignancy process and distinguishing different tumor stages or clinical outcomes of individual patients. In this study, we have conducted in silico pathway activation analysis using Riemannian manifold (RiePath) toward pan-cancer personalized characterization, which is the first attempt to apply the Riemannian manifold theory to measure the extent of pathway dysregulation in individual patient on the tangent space of the Riemannian manifold. RiePath effectively integrates pathway and gene expression information, not only generating a relatively low-dimensional and biologically relevant representation, but also identifying a robust panel of biologically meaningful pathway signatures as biomarkers. The pan-cancer analysis across 16 cancer types reveals the capability of RiePath to evaluate pathway activation accurately and identify clinical outcome-related pathways. We believe that RiePath has the potential to provide new prospects in understanding the molecular mechanisms of complex diseases and may find broader applications in predicting biomarkers for other intricate diseases.

Does the brain behave like a (complex) network? I. Dynamics

Graph theory is now becoming a standard tool in system-level neuroscience. However, endowing observed brain anatomy and dynamics with a complex network structure does not entail that the brain actually works as a network. Asking whether the brain behaves as a network means asking whether network properties count. From the viewpoint of neurophysiology and, possibly, of brain physics, the most substantial issues a network structure may be instrumental in addressing relate to the influence of network properties on brain dynamics and to whether these properties ultimately explain some aspects of brain function. Here, we address the dynamical implications of complex network, examining which aspects and scales of brain activity may be understood to genuinely behave as a network. To do so, we first define the meaning of networkness, and analyse some of its implications. We then examine ways in which brain anatomy and dynamics can be endowed with a network structure and discuss possible ways in which network structure may be shown to represent a genuine organisational principle of brain activity, rather than just a convenient description of its anatomy and dynamics.

Ecological decoding of visual aesthetic preference with oscillatory electroencephalogram features-A mini-review

In today's digital information age, human exposure to visual artifacts has reached an unprecedented quasi-omnipresence. Some of these cultural artifacts are elevated to the status of artworks which indicates a special appreciation of these objects. For many persons, the perception of such artworks coincides with aesthetic experiences (AE) that can positively affect health and wellbeing. AEs are composed of complex cognitive and affective mental and physiological states. More profound scientific understanding of the neural dynamics behind AEs would allow the development of passive Brain-Computer-Interfaces (BCI) that offer personalized art presentation to improve AE without the necessity of explicit user feedback. However, previous empirical research in visual neuroaesthetics predominantly investigated functional Magnetic Resonance Imaging and Event-Related-Potentials correlates of AE in unnaturalistic laboratory conditions which might not be the best features for practical neuroaesthetic BCIs. Furthermore, AE has, until recently, largely been framed as the experience of beauty or pleasantness. Yet, these concepts do not encompass all types of AE. Thus, the scope of these concepts is too narrow to allow personalized and optimal art experience across individuals and cultures. This narrative mini-review summarizes the state-of-the-art in oscillatory Electroencephalography (EEG) based visual neuroaesthetics and paints a road map toward the development of ecologically valid neuroaesthetic passive BCI systems that could optimize AEs, as well as their beneficial consequences. We detail reported oscillatory EEG correlates of AEs, as well as machine learning approaches to classify AE. We also highlight current limitations in neuroaesthetics and suggest future directions to improve EEG decoding of AE.

Exploring new horizons in neuroscience disease detection through innovative visual signal analysis

Brain disorders pose a substantial global health challenge, persisting as a leading cause of mortality worldwide. Electroencephalogram (EEG) analysis is crucial for diagnosing brain disorders, but it can be challenging for medical practitioners to interpret complex EEG signals and make accurate diagnoses. To address this, our study focuses on visualizing complex EEG signals in a format easily understandable by medical professionals and deep learning algorithms. We propose a novel time-frequency (TF) transform called the Forward-Backward Fourier transform (FBFT) and utilize convolutional neural networks (CNNs) to extract meaningful features from TF images and classify brain disorders. We introduce the concept of eye-naked classification, which integrates domain-specific knowledge and clinical expertise into the classification process. Our study demonstrates the effectiveness of the FBFT method, achieving impressive accuracies across multiple brain disorders using CNN-based classification. Specifically, we achieve accuracies of 99.82% for epilepsy, 95.91% for Alzheimer's disease (AD), 85.1% for murmur, and 100% for mental stress using CNN-based classification. Furthermore, in the context of naked-eye classification, we achieve accuracies of 78.6%, 71.9%, 82.7%, and 91.0% for epilepsy, AD, murmur, and mental stress, respectively. Additionally, we incorporate a mean correlation coefficient (mCC) based channel selection method to enhance the accuracy of our classification further. By combining these innovative approaches, our study enhances the visualization of EEG signals, providing medical professionals with a deeper understanding of TF medical images. This research has the potential to bridge the gap between image classification and visual medical interpretation, leading to better disease detection and improved patient care in the field of neuroscience.

Modeling Complex EEG Data Distribution on the Riemannian Manifold Toward Outlier Detection and Multimodal Classification

OBJECTIVE

The usage of Riemannian geometry for Brain-computer interfaces (BCIs) has gained momentum in recent years. Most of the machine learning techniques proposed for Riemannian BCIs consider the data distribution on a manifold to be unimodal. However, the distribution is likely to be multimodal rather than unimodal since high-data variability is a crucial limitation of electroencephalography (EEG). In this paper, we propose a novel data modeling method for considering complex data distributions on a Riemannian manifold of EEG covariance matrices, aiming to improve BCI reliability.

METHODS

Our method, Riemannian spectral clustering (RiSC), represents EEG covariance matrix distribution on a manifold using a graph with proposed similarity measurement based on geodesic distances, then clusters the graph nodes through spectral clustering. This allows flexibility to model both a unimodal and a multimodal distribution on a manifold. RiSC can be used as a basis to design an outlier detector named outlier detection Riemannian spectral clustering (odenRiSC) and a multimodal classifier named multimodal classifier Riemannian spectral clustering (mcRiSC). All required parameters of odenRiSC/mcRiSC are selected in data-driven manner. Moreover, there is no need to pre-set a threshold for outlier detection and the number of modes for multimodal classification.

RESULTS

The experimental evaluation revealed odenRiSC can detect EEG outliers more accurately than existing methods and mcRiSC outperformed the standard unimodal classifier, especially on high-variability datasets.

CONCLUSION

odenRiSC/mcRiSC are anticipated to contribute to making real-life BCIs outside labs and neuroergonomics applications more robust.

SIGNIFICANCE

RiSC can work as a robust EEG outlier detector and multimodal classifier.

A comprehensive dataset for home appliance control using ERP-based BCIs with the application of inter-subject transfer learning

Brain-computer interfaces (BCIs) have a potential to revolutionize human-computer interaction by enabling direct links between the brain and computer systems. Recent studies are increasingly focusing on practical applications of BCIs-e.g., home appliance control just by thoughts. One of the non-invasive BCIs using electroencephalography (EEG) capitalizes on event-related potentials (ERPs) in response to target stimuli and have shown promise in controlling home appliance. In this paper, we present a comprehensive dataset of online ERP-based BCIs for controlling various home appliances in diverse stimulus presentation environments. We collected online BCI data from a total of 84 subjects among whom 60 subjects controlled three types of appliances (TV: 30, door lock: 15, and electric light: 15) with 4 functions per appliance, 14 subjects controlled a Bluetooth speaker with 6 functions via an LCD monitor, and 10 subjects controlled air conditioner with 4 functions via augmented reality (AR). Using the dataset, we aimed to address the issue of inter-subject variability in ERPs by employing the transfer learning in two different approaches. The first approach, "within-paradigm transfer learning," aimed to generalize the model within the same paradigm of stimulus presentation. The second approach, "cross-paradigm transfer learning," involved extending the model from a 4-class LCD environment to different paradigms. The results demonstrated that transfer learning can effectively enhance the generalizability of BCIs based on ERP across different subjects and environments.

AR-Enabled Persistent Human-Machine Interfaces via a Scalable Soft Electrode Array

Augmented reality (AR) is a computer graphics technique that creates a seamless interface between the real and virtual worlds. AR usage rapidly spreads across diverse areas, such as healthcare, education, and entertainment. Despite its immense potential, AR interface controls rely on an external joystick, a smartphone, or a fixed camera system susceptible to lighting. Here, an AR-integrated soft wearable electronic system that detects the gestures of a subject for more intuitive, accurate, and direct control of external systems is introduced. Specifically, a soft, all-in-one wearable device includes a scalable electrode array and integrated wireless system to measure electromyograms for real-time continuous recognition of hand gestures. An advanced machine learning algorithm embedded in the system enables the classification of ten different classes with an accuracy of 96.08%. Compared to the conventional rigid wearables, the multi-channel soft wearable system offers an enhanced signal-to-noise ratio and consistency over multiple uses due to skin conformality. The demonstration of the AR-integrated soft wearable system for drone control captures the potential of the platform technology to offer numerous human-machine interface opportunities for users to interact remotely with external hardware and software.

Beta bursts question the ruling power for brain-computer interfaces

Objective: Current efforts to build reliable brain-computer interfaces (BCI) span multiple axes from hardware, to software, to more sophisticated experimental protocols, and personalized approaches. However, despite these abundant efforts, there is still room for significant improvement. We argue that a rather overlooked direction lies in linking BCI protocols with recent advances in fundamental neuroscience.Approach: In light of these advances, and particularly the characterization of the burst-like nature of beta frequency band activity and the diversity of beta bursts, we revisit the role of beta activity in 'left vs. right hand' motor imagery (MI) tasks. Current decoding approaches for such tasks take advantage of the fact that MI generates time-locked changes in induced power in the sensorimotor cortex and rely on band-passed power changes in single or multiple channels. Although little is known about the dynamics of beta burst activity during MI, we hypothesized that beta bursts should be modulated in a way analogous to their activity during performance of real upper limb movements.Main results and Significance: We show that classification features based on patterns of beta burst modulations yield decoding results that are equivalent to or better than typically used beta power across multiple open electroencephalography datasets, thus providing insights into the specificity of these bio-markers.

A study on the combination of functional connection features and Riemannian manifold in EEG emotion recognition

Introduction

Affective computing is the core for Human-computer interface (HCI) to be more intelligent, where electroencephalogram (EEG) based emotion recognition is one of the primary research orientations. Besides, in the field of brain-computer interface, Riemannian manifold is a highly robust and effective method. However, the symmetric positive definiteness (SPD) of the features limits its application.

Methods

In the present work, we introduced the Laplace matrix to transform the functional connection features, i.e., phase locking value (PLV), Pearson correlation coefficient (PCC), spectral coherent (COH), and mutual information (MI), to into semi-positive, and the max operator to ensure the transformed feature be positive. Then the SPD network is employed to extract the deep spatial information and a fully connected layer is employed to validate the effectiveness of the extracted features. Particularly, the decision layer fusion strategy is utilized to achieve more accurate and stable recognition results, and the differences of classification performance of different feature combinations are studied. What's more, the optimal threshold value applied to the functional connection feature is also studied.

Results

The public emotional dataset, SEED, is adopted to test the proposed method with subject dependent cross-validation strategy. The result of average accuracies for the four features indicate that PCC outperform others three features. The proposed model achieve best accuracy of 91.05% for the fusion of PLV, PCC, and COH, followed by the fusion of all four features with the accuracy of 90.16%.

Discussion

The experimental results demonstrate that the optimal thresholds for the four functional connection features always kept relatively stable within a fixed interval. In conclusion, the experimental results demonstrated the effectiveness of the proposed method.

Towards user-centric BCI design: Markov chain-based user assessment for mental imagery EEG-BCIs

Objective.While electroencephalography (EEG)-based brain-computer interfaces (BCIs) have many potential clinical applications, their use is impeded by poor performance for many users. To improve BCI performance, either via enhanced signal processing or user training, it is critical to understand and describe each user's ability to perform mental control tasks and produce discernible EEG patterns. While classification accuracy has predominantly been used to assess user performance, limitations and criticisms of this approach have emerged, thus prompting the need to develop novel user assessment approaches with greater descriptive capability. Here, we propose a combination of unsupervised clustering and Markov chain models to assess and describe user skill.Approach.Using unsupervisedK-means clustering, we segmented the EEG signal space into regions representing pattern states that users could produce. A user's movement through these pattern states while performing different tasks was modeled using Markov chains. Finally, using the steady-state distributions and entropy rates of the Markov chains, we proposed two metricstaskDistinctandrelativeTaskInconsistencyto assess, respectively, a user's ability to (i) produce distinct task-specific patterns for each mental task and (ii) maintain consistent patterns during individual tasks.Main results.Analysis of data from 14 adolescents using a three-class BCI revealed significant correlations between thetaskDistinctandrelativeTaskInconsistencymetrics and classification F1 score. Moreover, analysis of the pattern states and Markov chain models yielded descriptive information regarding user performance not immediately apparent from classification accuracy.Significance.Our proposed user assessment method can be used in concert with classifier-based analysis to further understand the extent to which users produce task-specific, time-evolving EEG patterns. In turn, this information could be used to enhance user training or classifier design.

Detection of tactile-based error-related potentials (ErrPs) in human-robot interaction

Robot learning based on implicitly extracted error detections (e.g., EEG-based error detections) has been well-investigated in human-robot interaction (HRI). In particular, the use of error-related potential (ErrP) evoked when recognizing errors is advantageous for robot learning when evaluation criteria cannot be explicitly defined, e.g., due to the complex behavior of robots. In most studies, erroneous behavior of robots were recognized visually. In some studies, visuo-tactile stimuli were used to evoke ErrPs or a tactile cue was used to indicate upcoming errors. To our knowledge, there are no studies in which ErrPs are evoked when recognizing errors only via the tactile channel. Hence, we investigated ErrPs evoked by tactile recognition of errors during HRI. In our scenario, subjects recognized errors caused by incorrect behavior of an orthosis during the execution of arm movements tactilely. EEG data from eight subjects was recorded. Subjects were asked to give a motor response to ensure error detection. Latency between the occurrence of errors and the response to errors was expected to be short. We assumed that the motor related brain activity is timely correlated with the ErrP and might be used from the classifier. To better interpret and test our results, we therefore tested ErrP detections in two additional scenarios, i.e., without motor response and with delayed motor response. In addition, we transferred three scenarios (motor response, no motor response, delayed motor response). Response times to error was short. However, high ErrP-classification performance was found for all subjects in case of motor response and no motor response condition. Further, ErrP classification performance was reduced for the transfer between motor response and delayed motor response, but not for the transfer between motor response and no motor response. We have shown that tactilely induced errors can be detected with high accuracy from brain activity. Our preliminary results suggest that also in tactile ErrPs the brain response is clear enough such that motor response is not relevant for classification. However, in future work, we will more systematically investigate tactile-based ErrP classification.

Classification of Targets and Distractors in an Audiovisual Attention Task Based on Electroencephalography

Within the broader context of improving interactions between artificial intelligence and humans, the question has arisen regarding whether auditory and rhythmic support could increase attention for visual stimuli that do not stand out clearly from an information stream. To this end, we designed an experiment inspired by pip-and-pop but more appropriate for eliciting attention and P3a-event-related potentials (ERPs). In this study, the aim was to distinguish between targets and distractors based on the subject's electroencephalography (EEG) data. We achieved this objective by employing different machine learning (ML) methods for both individual-subject (IS) and cross-subject (CS) models. Finally, we investigated which EEG channels and time points were used by the model to make its predictions using saliency maps. We were able to successfully perform the aforementioned classification task for both the IS and CS scenarios, reaching classification accuracies up to 76%. In accordance with the literature, the model primarily used the parietal-occipital electrodes between 200 ms and 300 ms after the stimulus to make its prediction. The findings from this research contribute to the development of more effective P300-based brain-computer interfaces. Furthermore, they validate the EEG data collected in our experiment.

Tensor-CSPNet: A Novel Geometric Deep Learning Framework for Motor Imagery Classification

Deep learning (DL) has been widely investigated in a vast majority of applications in electroencephalography (EEG)-based brain-computer interfaces (BCIs), especially for motor imagery (MI) classification in the past five years. The mainstream DL methodology for the MI-EEG classification exploits the temporospatial patterns of EEG signals using convolutional neural networks (CNNs), which have been particularly successful in visual images. However, since the statistical characteristics of visual images depart radically from EEG signals, a natural question arises whether an alternative network architecture exists apart from CNNs. To address this question, we propose a novel geometric DL (GDL) framework called Tensor-CSPNet, which characterizes spatial covariance matrices derived from EEG signals on symmetric positive definite (SPD) manifolds and fully captures the temporospatiofrequency patterns using existing deep neural networks on SPD manifolds, integrating with experiences from many successful MI-EEG classifiers to optimize the framework. In the experiments, Tensor-CSPNet attains or slightly outperforms the current state-of-the-art performance on the cross-validation and holdout scenarios in two commonly used MI-EEG datasets. Moreover, the visualization and interpretability analyses also exhibit the validity of Tensor-CSPNet for the MI-EEG classification. To conclude, in this study, we provide a feasible answer to the question by generalizing the DL methodologies on SPD manifolds, which indicates the start of a specific GDL methodology for the MI-EEG classification.

Improving EEG-based decoding of the locus of auditory attention through domain adaptation. J Neural Eng 2023;20:066022. [PMID: 37988748 DOI: 10.1088/1741-2552/ad0e7b] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Objective.This paper presents a novel domain adaptation (DA) framework to enhance the accuracy of electroencephalography (EEG)-based auditory attention classification, specifically for classifying the direction (left or right) of attended speech. The framework aims to improve the performances for subjects with initially low classification accuracy, overcoming challenges posed by instrumental and human factors. Limited dataset size, variations in EEG data quality due to factors such as noise, electrode misplacement or subjects, and the need for generalization across different trials, conditions and subjects necessitate the use of DA methods. By leveraging DA methods, the framework can learn from one EEG dataset and adapt to another, potentially resulting in more reliable and robust classification models.Approach.This paper focuses on investigating a DA method, based on parallel transport, for addressing the auditory attention classification problem. The EEG data utilized in this study originates from an experiment where subjects were instructed to selectively attend to one of the two spatially separated voices presented simultaneously.Main results.Significant improvement in classification accuracy was observed when poor data from one subject was transported to the domain of good data from different subjects, as compared to the baseline. The mean classification accuracy for subjects with poor data increased from 45.84% to 67.92%. Specifically, the highest achieved classification accuracy from one subject reached 83.33%, a substantial increase from the baseline accuracy of 43.33%.Significance.The findings of our study demonstrate the improved classification performances achieved through the implementation of DA methods. This brings us a step closer to leveraging EEG in neuro-steered hearing devices.

Combining brain-computer interfaces with deep reinforcement learning for robot training: a feasibility study in a simulation environment

Deep reinforcement learning (RL) is used as a strategy to teach robot agents how to autonomously learn complex tasks. While sparsity is a natural way to define a reward in realistic robot scenarios, it provides poor learning signals for the agent, thus making the design of good reward functions challenging. To overcome this challenge learning from human feedback through an implicit brain-computer interface (BCI) is used. We combined a BCI with deep RL for robot training in a 3-D physical realistic simulation environment. In a first study, we compared the feasibility of different electroencephalography (EEG) systems (wet- vs. dry-based electrodes) and its application for automatic classification of perceived errors during a robot task with different machine learning models. In a second study, we compared the performance of the BCI-based deep RL training to feedback explicitly given by participants. Our findings from the first study indicate the use of a high-quality dry-based EEG-system can provide a robust and fast method for automatically assessing robot behavior using a sophisticated convolutional neural network machine learning model. The results of our second study prove that the implicit BCI-based deep RL version in combination with the dry EEG-system can significantly accelerate the learning process in a realistic 3-D robot simulation environment. Performance of the BCI-based trained deep RL model was even comparable to that achieved by the approach with explicit human feedback. Our findings emphasize the usage of BCI-based deep RL methods as a valid alternative in those human-robot applications where no access to cognitive demanding explicit human feedback is available.

Riemannian geometric and ensemble learning for decoding cross-session motor imagery electroencephalography signals

Objective.Brain-computer interfaces (BCIs) enable a direct communication pathway between the human brain and external devices, without relying on the traditional peripheral nervous and musculoskeletal systems. Motor imagery (MI)-based BCIs have attracted significant interest for their potential in motor rehabilitation. However, current algorithms fail to account for the cross-session variability of electroencephalography signals, limiting their practical application.Approach.We proposed a Riemannian geometry-based adaptive boosting and voting ensemble (RAVE) algorithm to address this issue. Our approach segmented the MI period into multiple sub-datasets using a sliding window approach and extracted features from each sub-dataset using Riemannian geometry. We then trained adaptive boosting (AdaBoost) ensemble learning classifiers for each sub-dataset, with the final BCI output determined by majority voting of all classifiers. We tested our proposed RAVE algorithm and eight other competing algorithms on four datasets (Pan2023, BNCI001-2014, BNCI001-2015, BNCI004-2015).Main results.Our results showed that, in the cross-session scenario, the RAVE algorithm outperformed the eight other competing algorithms significantly under different within-session training sample sizes. Compared to traditional algorithms that involved a large number of training samples, the RAVE algorithm achieved similar or even better classification performance on the datasets (Pan2023, BNCI001-2014, BNCI001-2015), even when it did not use or only used a small number of within-session training samples.Significance.These findings indicate that our cross-session decoding strategy could enable MI-BCI applications that require no or minimal training process.

Enhanced Motor Imagery Decoding by Calibration Model-Assisted With Tactile ERD

OBJECTIVE

In this study, we propose a tactile-assisted calibration method for a motor imagery (MI) based Brain-Computer Interface (BCI) system.

METHOD

In the proposed calibration, tactile stimulation was applied to the hand wrist to assist the subjects in the MI task, which is named SA-MI task. Then, classifier training in the SA-MI Calibration was performed using the SA-MI data, while the Conventional Calibration employed the MI data. After the classifiers were trained, the performance was evaluated on a common MI dataset.

RESULTS

Our study demonstrated that the SA-MI Calibration significantly improved the performance as compared with the Conventional Calibration, with a decoding accuracy of (78.3% vs. 71.3%). Moreover, the average calibration time could be reduced by 40%. This benefit of the SA-MI Calibration effect was further validated by an independent control group, which showed no improvement when tactile stimulation was not applied during the calibration phase. Further analysis showed that when compared with MI, greater motor-related cortical activation and higher R 2 value in the alpha-beta frequency band were induced in SA-MI.

CONCLUSION

Indeed, the SA-MI Calibration could significantly improve the performance and reduce the calibration time as compared with the Conventional Calibration.

SIGNIFICANCE

The proposed tactile stimulation-assisted MI Calibration method holds great potential for a faster and more accurate system setup at the beginning of BCI usage.