1
|
Pfeffer MA, Ling SSH, Wong JKW. Exploring the frontier: Transformer-based models in EEG signal analysis for brain-computer interfaces. Comput Biol Med 2024; 178:108705. [PMID: 38865781 DOI: 10.1016/j.compbiomed.2024.108705] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2023] [Revised: 05/01/2024] [Accepted: 06/01/2024] [Indexed: 06/14/2024]
Abstract
This review systematically explores the application of transformer-based models in EEG signal processing and brain-computer interface (BCI) development, with a distinct focus on ensuring methodological rigour and adhering to empirical validations within the existing literature. By examining various transformer architectures, such as the Temporal Spatial Transformer Network (TSTN) and EEG Conformer, this review delineates their capabilities in mitigating challenges intrinsic to EEG data, such as noise and artifacts, and their subsequent implications on decoding and classification accuracies across disparate mental tasks. The analytical scope extends to a meticulous examination of attention mechanisms within transformer models, delineating their role in illuminating critical temporal and spatial EEG features and facilitating interpretability in model decision-making processes. The discourse additionally encapsulates emerging works that substantiate the efficacy of transformer models in noise reduction of EEG signals and diversifying applications beyond the conventional motor imagery paradigm. Furthermore, this review elucidates evident gaps and propounds exploratory avenues in the applications of pre-trained transformers in EEG analysis and the potential expansion into real-time and multi-task BCI applications. Collectively, this review distils extant knowledge, navigates through the empirical findings, and puts forward a structured synthesis, thereby serving as a conduit for informed future research endeavours in transformer-enhanced, EEG-based BCI systems.
Collapse
Affiliation(s)
- Maximilian Achim Pfeffer
- Faculty of Engineering and Information Technology, University of Technology Sydney, CB11 81-113, Broadway, Ultimo, 2007, New South Wales, Australia.
| | - Steve Sai Ho Ling
- Faculty of Engineering and Information Technology, University of Technology Sydney, CB11 81-113, Broadway, Ultimo, 2007, New South Wales, Australia.
| | - Johnny Kwok Wai Wong
- Faculty of Design, Architecture and Building, University of Technology Sydney, 15 Broadway, Ultimo, 2007, New South Wales, Australia.
| |
Collapse
|
2
|
|
3
|
Kawala-Sterniuk A, Browarska N, Al-Bakri A, Pelc M, Zygarlicki J, Sidikova M, Martinek R, Gorzelanczyk EJ. Summary of over Fifty Years with Brain-Computer Interfaces-A Review. Brain Sci 2021; 11:43. [PMID: 33401571 PMCID: PMC7824107 DOI: 10.3390/brainsci11010043] [Citation(s) in RCA: 47] [Impact Index Per Article: 15.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2020] [Revised: 12/25/2020] [Accepted: 12/27/2020] [Indexed: 11/16/2022] Open
Abstract
Over the last few decades, the Brain-Computer Interfaces have been gradually making their way to the epicenter of scientific interest. Many scientists from all around the world have contributed to the state of the art in this scientific domain by developing numerous tools and methods for brain signal acquisition and processing. Such a spectacular progress would not be achievable without accompanying technological development to equip the researchers with the proper devices providing what is absolutely necessary for any kind of discovery as the core of every analysis: the data reflecting the brain activity. The common effort has resulted in pushing the whole domain to the point where the communication between a human being and the external world through BCI interfaces is no longer science fiction but nowadays reality. In this work we present the most relevant aspects of the BCIs and all the milestones that have been made over nearly 50-year history of this research domain. We mention people who were pioneers in this area as well as we highlight all the technological and methodological advances that have transformed something available and understandable by a very few into something that has a potential to be a breathtaking change for so many. Aiming to fully understand how the human brain works is a very ambitious goal and it will surely take time to succeed. However, even that fraction of what has already been determined is sufficient e.g., to allow impaired people to regain control on their lives and significantly improve its quality. The more is discovered in this domain, the more benefit for all of us this can potentially bring.
Collapse
Affiliation(s)
- Aleksandra Kawala-Sterniuk
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, 45-758 Opole, Poland; (N.B.); (M.P.); (J.Z.)
| | - Natalia Browarska
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, 45-758 Opole, Poland; (N.B.); (M.P.); (J.Z.)
| | - Amir Al-Bakri
- Department of Biomedical Engineering, College of Engineering, University of Babylon, 51001 Babylon, Iraq;
| | - Mariusz Pelc
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, 45-758 Opole, Poland; (N.B.); (M.P.); (J.Z.)
- Department of Computing and Information Systems, University of Greenwich, London SE10 9LS, UK
| | - Jaroslaw Zygarlicki
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, 45-758 Opole, Poland; (N.B.); (M.P.); (J.Z.)
| | - Michaela Sidikova
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University Ostrava—FEECS, 708 00 Ostrava-Poruba, Czech Republic; (M.S.); (R.M.)
| | - Radek Martinek
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University Ostrava—FEECS, 708 00 Ostrava-Poruba, Czech Republic; (M.S.); (R.M.)
| | - Edward Jacek Gorzelanczyk
- Department of Theoretical Basis of BioMedical Sciences and Medical Informatics, Nicolaus Copernicus University, Collegium Medicum, 85-067 Bydgoszcz, Poland;
- Institute of Philosophy, Kazimierz Wielki University, 85-092 Bydgoszcz, Poland
- Babinski Specialist Psychiatric Healthcare Center, Outpatient Addiction Treatment, 91-229 Lodz, Poland
- The Society for the Substitution Treatment of Addiction “Medically Assisted Recovery”, 85-791 Bydgoszcz, Poland
| |
Collapse
|
4
|
Mental imagery classification using one-dimensional convolutional neural network for target selection in single-channel BCI-controlled mobile robot. Neural Comput Appl 2020. [DOI: 10.1007/s00521-020-05393-6] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
5
|
A comprehensive assessment of Brain Computer Interfaces: Recent trends and challenges. J Neurosci Methods 2020; 346:108918. [PMID: 32853592 DOI: 10.1016/j.jneumeth.2020.108918] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2020] [Revised: 07/15/2020] [Accepted: 08/19/2020] [Indexed: 12/23/2022]
Abstract
BACKGROUND An uninterrupted channel of communication and control between the human brain and electronic processing units has led to an increased use of Brain Computer Interfaces (BCIs). This article attempts to present an all-encompassing review on BCI and the scientific advancements associated with it. The ultimate goal of this review is to provide a general overview of the BCI technology and to shed light on different aspects of BCIs. This review also underscores the applications, practical challenges and opportunities associated with BCI technology, which can be used to accelerate future developments in this field. METHODS This review is based on a systematic literature search for tracking down the relevant research annals and proceedings. Using a methodical search strategy, the search was carried out across major technical databases. The retrieved records were screened for their relevance and a total of 369 research chronicles were engulfed in this review based on the inclusion criteria. RESULTS This review describes the present scenario and recent advancements in BCI technology. It also identifies several application areas of BCI technology. This comprehensive review provides evidence that, while we are getting ever closer, significant challenges still exist for the development of BCIs that can seamlessly integrate with the user's biological system. CONCLUSION The findings of this review confirm the importance of BCI technology in various applications. It is concluded that BCI technology, still in its sprouting phase, requires significant explorations for further development.
Collapse
|
6
|
Karkosz S, Jukiewicz M. Feature selection for classification in Steady state visually evoked potentials (SSVEP)-based brain-computer interfaces with genetic algorithm. BIO-ALGORITHMS AND MED-SYSTEMS 2020. [DOI: 10.1515/bams-2020-0013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Abstract
Objectives
Optimization of Brain-Computer Interface by detecting the minimal number of morphological features of signal that maximize accuracy.
Methods
System of signal processing and morphological features extractor was designed, then the genetic algorithm was used to select such characteristics that maximize the accuracy of the signal’s frequency recognition in offline Brain-Computer Interface (BCI).
Results
The designed system provides higher accuracy results than a previously developed system that uses the same preprocessing methods, however, different results were achieved for various subjects.
Conclusions
It is possible to enhance the previously developed BCI by combining it with morphological features extraction, however, it’s performance is dependent on subject variability.
Collapse
Affiliation(s)
- Stanisław Karkosz
- SWPS University of Social Sciences and Humanities , Warszawa , Poland
| | | |
Collapse
|
7
|
Abstract
The electroencephalogram (EEG) was invented almost 100 years ago and is still a method of choice for many research questions, even applications-from functional brain imaging in neuroscientific investigations during movement to real-time applications like brain-computer interfacing. This chapter gives some background information on the establishment and properties of the EEG. This chapter starts with a closer look at the sources of EEG at a micro or neuronal level, followed by recording techniques, types of electrodes, and common EEG artifacts. Then an overview on EEG phenomena, namely, spontaneous EEG and event-related potentials build the middle part of this chapter. The last part discusses brain signals, which are used in current BCI research, including short descriptions and examples of applications.
Collapse
Affiliation(s)
- Gernot R Müller-Putz
- Institute for Neural Engineering, Laboratory of Brain-Computer Interfaces, Graz University of Technology, Graz, Austria.
| |
Collapse
|
8
|
Mohino-Herranz I, Gil-Pita R, Rosa-Zurera M, Seoane F. Activity Recognition Using Wearable Physiological Measurements: Selection of Features from a Comprehensive Literature Study. SENSORS (BASEL, SWITZERLAND) 2019; 19:E5524. [PMID: 31847261 PMCID: PMC6960825 DOI: 10.3390/s19245524] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/12/2019] [Revised: 12/07/2019] [Accepted: 12/11/2019] [Indexed: 11/17/2022]
Abstract
Activity and emotion recognition based on physiological signal processing in health care applications is a relevant research field, with promising future and relevant applications, such as health at work or preventive care. This paper carries out a deep analysis of features proposed to extract information from the electrocardiogram, thoracic electrical bioimpedance, and electrodermal activity signals. The activities analyzed are: neutral, emotional, mental and physical. A total number of 533 features are tested for activity recognition, performing a comprehensive study taking into consideration the prediction accuracy, feature calculation, window length, and type of classifier. Feature selection to know the most relevant features from the complete set is implemented using a genetic algorithm, with a different number of features. This study has allowed us to determine the best number of features to obtain a good error probability avoiding over-fitting, and the best subset of features among those proposed in the literature. The lowest error probability that is obtained is 22.2%, with 40 features, a least squares error classifier, and 40 seconds window length.
Collapse
Affiliation(s)
- Inma Mohino-Herranz
- Department of Signal Theory and Communications, University of Alcala, 28805 Alcala de Henares, Madrid, Spain; (R.G.-P.); (M.R.-Z.)
| | - Roberto Gil-Pita
- Department of Signal Theory and Communications, University of Alcala, 28805 Alcala de Henares, Madrid, Spain; (R.G.-P.); (M.R.-Z.)
| | - Manuel Rosa-Zurera
- Department of Signal Theory and Communications, University of Alcala, 28805 Alcala de Henares, Madrid, Spain; (R.G.-P.); (M.R.-Z.)
| | - Fernando Seoane
- Clinical Science, Intervention an Technology, Karolinska Institutet, 17177 Stockholm, Sweden;
- Department Biomedical Engineering, Karolinska University Hospital, 14186 Stockholm, Sweden
- Swedish School of Textiles, University of Boras, 50190 Boras, Sweden
| |
Collapse
|
9
|
Eliseyev A, Aksenova T. Personalized adaptive instruction design (PAID) for brain–computer interface using reinforcement learning and deep learning: simulated data study. BRAIN-COMPUTER INTERFACES 2019. [DOI: 10.1080/2326263x.2019.1651570] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
Affiliation(s)
- A. Eliseyev
- University Grenoble Alpes, CEA, LETI, CLINATEC, Grenoble, France
| | - T. Aksenova
- University Grenoble Alpes, CEA, LETI, CLINATEC, Grenoble, France
| |
Collapse
|
10
|
Schneider C, Pereira M, Tonin L, Millán JDR. Real-time EEG Feedback on Alpha Power Lateralization Leads to Behavioral Improvements in a Covert Attention Task. Brain Topogr 2019; 33:48-59. [PMID: 31317285 DOI: 10.1007/s10548-019-00725-9] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2019] [Accepted: 07/02/2019] [Indexed: 01/10/2023]
Abstract
Visual attention can be spatially oriented, even in the absence of saccadic eye-movements, to facilitate the processing of incoming visual information. One behavioral proxy for this so-called covert visuospatial attention (CVSA) is the validity effect (VE): the reduction in reaction time (RT) to visual stimuli at attended locations and the increase in RT to stimuli at unattended locations. At the electrophysiological level, one correlate of CVSA is the lateralization in the occipital [Formula: see text]-band oscillations, resulting from [Formula: see text]-power increases ipsilateral and decreases contralateral to the attended hemifield. While this [Formula: see text]-band lateralization has been considerably studied using electroencephalography (EEG) or magnetoencephalography (MEG), little is known about whether it can be trained to improve CVSA behaviorally. In this cross-over sham-controlled study we used continuous real-time feedback of the occipital [Formula: see text]-lateralization to modulate behavioral and electrophysiological markers of covert attention. Fourteen subjects performed a cued CVSA task, involving fast responses to covertly attended stimuli. During real-time feedback runs, trials extended in time if subjects reached states of high [Formula: see text]-lateralization. Crucially, the ongoing [Formula: see text]-lateralization was fed back to the subject by changing the color of the attended stimulus. We hypothesized that this ability to self-monitor lapses in CVSA and thus being able to refocus attention accordingly would lead to improved CVSA performance during subsequent testing. We probed the effect of the intervention by evaluating the pre-post changes in the VE and the [Formula: see text]-lateralization. Behaviorally, results showed a significant interaction between feedback (experimental-sham) and time (pre-post) for the validity effect, with an increase in performance only for the experimental condition. We did not find corresponding pre-post changes in the [Formula: see text]-lateralization. Our findings suggest that EEG-based real-time feedback is a promising tool to enhance the level of covert visuospatial attention, especially with respect to behavioral changes. This opens up the exploration of applications of the proposed training method for the cognitive rehabilitation of attentional disorders.
Collapse
Affiliation(s)
- Christoph Schneider
- Chair for Brain-Machine Interface (CNBI), École Polytechnique Fédérale de Lausanne (EPFL), 1202, Geneva, Switzerland.
| | - Michael Pereira
- Chair for Brain-Machine Interface (CNBI), École Polytechnique Fédérale de Lausanne (EPFL), 1202, Geneva, Switzerland.,Laboratory of Cognitive Neuroscience (LNCO), École Polytechnique Fédérale de Lausanne (EPFL), 1202, Geneva, Switzerland
| | - Luca Tonin
- Chair for Brain-Machine Interface (CNBI), École Polytechnique Fédérale de Lausanne (EPFL), 1202, Geneva, Switzerland
| | - José Del R Millán
- Chair for Brain-Machine Interface (CNBI), École Polytechnique Fédérale de Lausanne (EPFL), 1202, Geneva, Switzerland.
| |
Collapse
|
11
|
Yousefi R, Rezazadeh Sereshkeh A, Chau T. Online detection of error-related potentials in multi-class cognitive task-based BCIs. BRAIN-COMPUTER INTERFACES 2019. [DOI: 10.1080/2326263x.2019.1614770] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Affiliation(s)
- Rozhin Yousefi
- Institute of Biomaterials and Biomedical Engineering, University of Toronto, Toronto, Canada
| | | | - Tom Chau
- Institute of Biomaterials and Biomedical Engineering, University of Toronto, Toronto, Canada
- Bloorview Research Institute, Holland Bloorview Hospital, Toronto, Canada
| |
Collapse
|
12
|
Yousefi R, Sereshkeh AR, Chau T. Exploiting error-related potentials in cognitive task based BCI. Biomed Phys Eng Express 2018. [DOI: 10.1088/2057-1976/aaee99] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
13
|
Schaeffer MC, Aksenova T. Data-Driven Transducer Design and Identification for Internally-Paced Motor Brain Computer Interfaces: A Review. Front Neurosci 2018; 12:540. [PMID: 30158847 PMCID: PMC6104172 DOI: 10.3389/fnins.2018.00540] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2017] [Accepted: 07/17/2018] [Indexed: 11/13/2022] Open
Abstract
Brain-Computer Interfaces (BCIs) are systems that establish a direct communication pathway between the users' brain activity and external effectors. They offer the potential to improve the quality of life of motor-impaired patients. Motor BCIs aim to permit severely motor-impaired users to regain limb mobility by controlling orthoses or prostheses. In particular, motor BCI systems benefit patients if the decoded actions reflect the users' intentions with an accuracy that enables them to efficiently interact with their environment. One of the main challenges of BCI systems is to adapt the BCI's signal translation blocks to the user to reach a high decoding accuracy. This paper will review the literature of data-driven and user-specific transducer design and identification approaches and it focuses on internally-paced motor BCIs. In particular, continuous kinematic biomimetic and mental-task decoders are reviewed. Furthermore, static and dynamic decoding approaches, linear and non-linear decoding, offline and real-time identification algorithms are considered. The current progress and challenges related to the design of clinical-compatible motor BCI transducers are additionally discussed.
Collapse
Affiliation(s)
| | - Tetiana Aksenova
- CEA, LETI, CLINATEC, MINATEC Campus, Université Grenoble Alpes, Grenoble, France
| |
Collapse
|
14
|
Mammone N, Ieracitano C, Adeli H, Bramanti A, Morabito FC. Permutation Jaccard Distance-Based Hierarchical Clustering to Estimate EEG Network Density Modifications in MCI Subjects. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2018; 29:5122-5135. [PMID: 29994428 DOI: 10.1109/tnnls.2018.2791644] [Citation(s) in RCA: 39] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
In this paper, a novel electroencephalographic (EEG)-based method is introduced for the quantification of brain-electrical connectivity changes over a longitudinal evaluation of mild cognitive impaired (MCI) subjects. In the proposed method, a dissimilarity matrix is constructed by estimating the coupling strength between every pair of EEG signals, Hierarchical clustering is then applied to group the related electrodes according to the dissimilarity estimated on pairs of EEG recordings. Subsequently, the connectivity density of the electrodes network is calculated. The technique was tested over two different coupling strength descriptors: wavelet coherence (WC) and permutation Jaccard distance (PJD), a novel metric of coupling strength between time series introduced in this paper. Twenty-five MCI patients were enrolled within a follow-up program that consisted of two successive evaluations, at time T0 and at time T1, three months later. At T1, four subjects were diagnosed to have converted to Alzheimer's Disease (AD). When applying the PJD-based method, the converted patients exhibited a significantly increased PJD (p < 0.05), i.e., a reduced overall coupling strength, specifically in delta and θ bands and in the overall range (0.5-32 Hz). In addition, in contrast to stable MCI patients, converted patients exhibited a network density reduction in every subband (delta, θ, alpha, and beta). When WC was used as coupling strength descriptor, the method resulted in a less sensitive and specific outcome. The proposed method, mixing nonlinear analysis to a machine learning approach, appears to provide an objective evaluation of the connectivity density modifications associated to the MCI-AD conversion, just processing noninvasive EEG signals.
Collapse
|
15
|
A Game Player Expertise Level Classification System Using Electroencephalography (EEG). APPLIED SCIENCES-BASEL 2017. [DOI: 10.3390/app8010018] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
16
|
Dhindsa K, Carcone D, Becker S. Toward an Open-Ended BCI: A User-Centered Coadaptive Design. Neural Comput 2017; 29:2742-2768. [PMID: 28777722 DOI: 10.1162/neco_a_01001] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Brain-computer interfaces (BCIs) allow users to control a device by interpreting their brain activity. For simplicity, these devices are designed to be operated by purposefully modulating specific predetermined neurophysiological signals, such as the sensorimotor rhythm. However, the ability to modulate a given neurophysiological signal is highly variable across individuals, contributing to the inconsistent performance of BCIs for different users. These differences suggest that individuals who experience poor BCI performance with one class of brain signals might have good results with another. In order to take advantage of individual abilities as they relate to BCI control, we need to move beyond the current approaches. In this letter, we explore a new BCI design aimed at a more individualized and user-focused experience, which we call open-ended BCI. Individual users were given the freedom to discover their own mental strategies as opposed to being trained to modulate a given brain signal. They then underwent multiple coadaptive training sessions with the BCI. Our first open-ended BCI performed similarly to comparable BCIs while accommodating a wider variety of mental strategies without a priori knowledge of the specific brain signals any individual might use. Post hoc analysis revealed individual differences in terms of which sensory modality yielded optimal performance. We found a large and significant effect of individual differences in background training and expertise, such as in musical training, on BCI performance. Future research should be focused on finding more generalized solutions to user training and brain state decoding methods to fully utilize the abilities of different individuals in an open-ended BCI. Accounting for each individual's areas of expertise could have important implications on BCI training and BCI application design.
Collapse
Affiliation(s)
- Kiret Dhindsa
- Neurotechnology and Neuroplasticity Lab, Department of Psychology, Neuroscience, and Behaviour, McMaster University, Hamilton, Ontario L8S 4L8, Canada
| | - Dean Carcone
- Neurotechnology and Neuroplasticity Lab, Department of Psychology, Neuroscience, and Behaviour, McMaster University, Hamilton, Ontario L8S 4L8, Canada
| | - Suzanna Becker
- Neurotechnology and Neuroplasticity Lab, Department of Psychology, Neuroscience, and Behaviour, McMaster University, Hamilton, Ontario L8S 4L8, Canada
| |
Collapse
|
17
|
Ludwig SA, Kong J. Investigation of different classifiers and channel configurations of a mobile P300-based brain-computer interface. Med Biol Eng Comput 2017; 55:2143-2154. [PMID: 28553694 DOI: 10.1007/s11517-017-1658-2] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2016] [Accepted: 05/13/2017] [Indexed: 12/01/2022]
Abstract
Innovative methods and new technologies have significantly improved the quality of our daily life. However, disabled people, for example those that cannot use their arms and legs anymore, often cannot benefit from these developments, since they cannot use their hands to interact with traditional interaction methods (such as mouse or keyboard) to communicate with a computer system. A brain-computer interface (BCI) system allows such a disabled person to control an external device via brain waves. Past research mostly dealt with static interfaces, which limit users to a stationary location. However, since we are living in a world that is highly mobile, this paper evaluates a speller interface on a mobile phone used in a moving condition. The spelling experiments were conducted with 14 able-bodied subjects using visual flashes as the stimulus to spell 47 alphanumeric characters (38 letters and 9 numbers). This data was then used for the classification experiments. In par- ticular, two research directions are pursued. The first investigates the impact of different classification algorithms, and the second direction looks at the channel configuration, i.e., which channels are most beneficial in terms of achieving the highest classification accuracy. The evaluation results indicate that the Bayesian Linear Discriminant Analysis algorithm achieves the best accuracy. Also, the findings of the investigation on the channel configuration, which can potentially reduce the amount of data processing on a mobile device with limited computing capacity, is especially useful in mobile BCIs.
Collapse
Affiliation(s)
- Simone A Ludwig
- Department of Computer Science, North Dakota State University, Fargo, ND, USA.
| | - Jun Kong
- Department of Computer Science, North Dakota State University, Fargo, ND, USA
| |
Collapse
|
18
|
Reducing training requirements through evolutionary based dimension reduction and subject transfer. Neurocomputing 2017. [DOI: 10.1016/j.neucom.2016.10.055] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
19
|
Zhang Y, Zhou G, Jin J, Zhao Q, Wang X, Cichocki A. Sparse Bayesian Classification of EEG for Brain-Computer Interface. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2016; 27:2256-2267. [PMID: 26415189 DOI: 10.1109/tnnls.2015.2476656] [Citation(s) in RCA: 122] [Impact Index Per Article: 15.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
Regularization has been one of the most popular approaches to prevent overfitting in electroencephalogram (EEG) classification of brain-computer interfaces (BCIs). The effectiveness of regularization is often highly dependent on the selection of regularization parameters that are typically determined by cross-validation (CV). However, the CV imposes two main limitations on BCIs: 1) a large amount of training data is required from the user and 2) it takes a relatively long time to calibrate the classifier. These limitations substantially deteriorate the system's practicability and may cause a user to be reluctant to use BCIs. In this paper, we introduce a sparse Bayesian method by exploiting Laplace priors, namely, SBLaplace, for EEG classification. A sparse discriminant vector is learned with a Laplace prior in a hierarchical fashion under a Bayesian evidence framework. All required model parameters are automatically estimated from training data without the need of CV. Extensive comparisons are carried out between the SBLaplace algorithm and several other competing methods based on two EEG data sets. The experimental results demonstrate that the SBLaplace algorithm achieves better overall performance than the competing algorithms for EEG classification.
Collapse
|
20
|
Advances in user-training for mental-imagery-based BCI control: Psychological and cognitive factors and their neural correlates. PROGRESS IN BRAIN RESEARCH 2016; 228:3-35. [PMID: 27590964 DOI: 10.1016/bs.pbr.2016.04.002] [Citation(s) in RCA: 59] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/08/2023]
Abstract
While being very promising for a wide range of applications, mental-imagery-based brain-computer interfaces (MI-BCIs) remain barely used outside laboratories, notably due to the difficulties users encounter when attempting to control them. Indeed, 10-30% of users are unable to control MI-BCIs (so-called BCI illiteracy) while only a small proportion reach acceptable control abilities. This huge interuser variability has led the community to investigate potential predictors of performance related to users' personality and cognitive profile. Based on a literature review, we propose a classification of these MI-BCI performance predictors into three categories representing high-level cognitive concepts: (1) users' relationship with the technology (including the notions of computer anxiety and sense of agency), (2) attention, and (3) spatial abilities. We detail these concepts and their neural correlates in order to better understand their relationship with MI-BCI user-training. Consequently, we propose, by way of future prospects, some guidelines to improve MI-BCI user-training.
Collapse
|
21
|
Yang B, Li H, Wang Q, Zhang Y. Subject-based feature extraction by using fisher WPD-CSP in brain-computer interfaces. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2016; 129:21-28. [PMID: 27084317 DOI: 10.1016/j.cmpb.2016.02.020] [Citation(s) in RCA: 39] [Impact Index Per Article: 4.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/03/2015] [Revised: 02/21/2016] [Accepted: 02/26/2016] [Indexed: 06/05/2023]
Abstract
BACKGROUND AND OBJECTIVE Feature extraction of electroencephalogram (EEG) plays a vital role in brain-computer interfaces (BCIs). In recent years, common spatial pattern (CSP) has been proven to be an effective feature extraction method. However, the traditional CSP has disadvantages of requiring a lot of input channels and the lack of frequency information. In order to remedy the defects of CSP, wavelet packet decomposition (WPD) and CSP are combined to extract effective features. But WPD-CSP method considers less about extracting specific features that are fitted for the specific subject. So a subject-based feature extraction method using fisher WPD-CSP is proposed in this paper. METHODS The idea of proposed method is to adapt fisher WPD-CSP to each subject separately. It mainly includes the following six steps: (1) original EEG signals from all channels are decomposed into a series of sub-bands using WPD; (2) average power values of obtained sub-bands are computed; (3) the specified sub-bands with larger values of fisher distance according to average power are selected for that particular subject; (4) each selected sub-band is reconstructed to be regarded as a new EEG channel; (5) all new EEG channels are used as input of the CSP and a six-dimensional feature vector is obtained by the CSP. The subject-based feature extraction model is so formed; (6) the probabilistic neural network (PNN) is used as the classifier and the classification accuracy is obtained. RESULTS Data from six subjects are processed by the subject-based fisher WPD-CSP, the non-subject-based fisher WPD-CSP and WPD-CSP, respectively. Compared with non-subject-based fisher WPD-CSP and WPD-CSP, the results show that the proposed method yields better performance (sensitivity: 88.7±0.9%, and specificity: 91±1%) and the classification accuracy from subject-based fisher WPD-CSP is increased by 6-12% and 14%, respectively. CONCLUSIONS The proposed subject-based fisher WPD-CSP method can not only remedy disadvantages of CSP by WPD but also discriminate helpless sub-bands for each subject and make remaining fewer sub-bands keep better separability by fisher distance, which leads to a higher classification accuracy than WPD-CSP method.
Collapse
Affiliation(s)
- Banghua Yang
- Department of Automation, School of Mechatronic Engineering and Automation; Shanghai Key Laboratory of Power Station Automation Technology, Shanghai University, Shanghai 200072, China.
| | - Huarong Li
- Department of Automation, School of Mechatronic Engineering and Automation; Shanghai Key Laboratory of Power Station Automation Technology, Shanghai University, Shanghai 200072, China
| | - Qian Wang
- Department of Automation, School of Mechatronic Engineering and Automation; Shanghai Key Laboratory of Power Station Automation Technology, Shanghai University, Shanghai 200072, China
| | - Yunyuan Zhang
- Department of Automation, School of Mechatronic Engineering and Automation; Shanghai Key Laboratory of Power Station Automation Technology, Shanghai University, Shanghai 200072, China
| |
Collapse
|
22
|
Jeunet C, Jahanpour E, Lotte F. Why standard brain-computer interface (BCI) training protocols should be changed: an experimental study. J Neural Eng 2016; 13:036024. [DOI: 10.1088/1741-2560/13/3/036024] [Citation(s) in RCA: 98] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
|
23
|
Agarwal SK, Shah S, Kumar R. Classification of mental tasks from EEG data using backtracking search optimization based neural classifier. Neurocomputing 2015. [DOI: 10.1016/j.neucom.2015.03.041] [Citation(s) in RCA: 31] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
24
|
Bhattacharyya S, Pal M, Konar A, Tibarewala D. An interval type-2 fuzzy approach for real-time EEG-based control of wrist and finger movement. Biomed Signal Process Control 2015. [DOI: 10.1016/j.bspc.2015.05.004] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
25
|
Estepp JR, Christensen JC. Electrode replacement does not affect classification accuracy in dual-session use of a passive brain-computer interface for assessing cognitive workload. Front Neurosci 2015; 9:54. [PMID: 25805963 PMCID: PMC4353251 DOI: 10.3389/fnins.2015.00054] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2014] [Accepted: 02/06/2015] [Indexed: 11/13/2022] Open
Abstract
The passive brain-computer interface (pBCI) framework has been shown to be a very promising construct for assessing cognitive and affective state in both individuals and teams. There is a growing body of work that focuses on solving the challenges of transitioning pBCI systems from the research laboratory environment to practical, everyday use. An interesting issue is what impact methodological variability may have on the ability to reliably identify (neuro)physiological patterns that are useful for state assessment. This work aimed at quantifying the effects of methodological variability in a pBCI design for detecting changes in cognitive workload. Specific focus was directed toward the effects of replacing electrodes over dual sessions (thus inducing changes in placement, electromechanical properties, and/or impedance between the electrode and skin surface) on the accuracy of several machine learning approaches in a binary classification problem. In investigating these methodological variables, it was determined that the removal and replacement of the electrode suite between sessions does not impact the accuracy of a number of learning approaches when trained on one session and tested on a second. This finding was confirmed by comparing to a control group for which the electrode suite was not replaced between sessions. This result suggests that sensors (both neurological and peripheral) may be removed and replaced over the course of many interactions with a pBCI system without affecting its performance. Future work on multi-session and multi-day pBCI system use should seek to replicate this (lack of) effect between sessions in other tasks, temporal time courses, and data analytic approaches while also focusing on non-stationarity and variable classification performance due to intrinsic factors.
Collapse
Affiliation(s)
- Justin R. Estepp
- Applied Neuroscience Branch, Human Effectiveness Directorate, 711th Human Performance Wing, Air Force Research LaboratoryWright-Patterson AFB, OH, USA
| | | |
Collapse
|
26
|
De Massari D, Pacheco D, Malekshahi R, Betella A, Verschure PFMJ, Birbaumer N, Caria A. Fast mental states decoding in mixed reality. Front Behav Neurosci 2014; 8:415. [PMID: 25505878 PMCID: PMC4245910 DOI: 10.3389/fnbeh.2014.00415] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2014] [Accepted: 11/12/2014] [Indexed: 11/16/2022] Open
Abstract
The combination of Brain-Computer Interface (BCI) technology, allowing online monitoring and decoding of brain activity, with virtual and mixed reality (MR) systems may help to shape and guide implicit and explicit learning using ecological scenarios. Real-time information of ongoing brain states acquired through BCI might be exploited for controlling data presentation in virtual environments. Brain states discrimination during mixed reality experience is thus critical for adapting specific data features to contingent brain activity. In this study we recorded electroencephalographic (EEG) data while participants experienced MR scenarios implemented through the eXperience Induction Machine (XIM). The XIM is a novel framework modeling the integration of a sensing system that evaluates and measures physiological and psychological states with a number of actuators and effectors that coherently reacts to the user's actions. We then assessed continuous EEG-based discrimination of spatial navigation, reading and calculation performed in MR, using linear discriminant analysis (LDA) and support vector machine (SVM) classifiers. Dynamic single trial classification showed high accuracy of LDA and SVM classifiers in detecting multiple brain states as well as in differentiating between high and low mental workload, using a 5 s time-window shifting every 200 ms. Our results indicate overall better performance of LDA with respect to SVM and suggest applicability of our approach in a BCI-controlled MR scenario. Ultimately, successful prediction of brain states might be used to drive adaptation of data representation in order to boost information processing in MR.
Collapse
Affiliation(s)
- Daniele De Massari
- Institut für Medizinische Psychologie und Verhaltensneurobiologie, Universität Tübingen Tübingen, Germany ; Fondazione Ospedale San Camillo, Istituto di Ricovero e Cura a Carattere Scientifico Venezia, Italy
| | - Daniel Pacheco
- SPECS - Laboratory of Synthetic Perceptive, Emotive and Cognitive Systems, Department of Technology, Center of Autonomous Systems and Neurorobotics, Universitat Pompeu Fabra Barcelona, Spain
| | - Rahim Malekshahi
- Institut für Medizinische Psychologie und Verhaltensneurobiologie, Universität Tübingen Tübingen, Germany ; Graduate School of Neural & Behavioural Sciences, International Max Planck Research School Tübingen, Germany
| | - Alberto Betella
- SPECS - Laboratory of Synthetic Perceptive, Emotive and Cognitive Systems, Department of Technology, Center of Autonomous Systems and Neurorobotics, Universitat Pompeu Fabra Barcelona, Spain
| | - Paul F M J Verschure
- SPECS - Laboratory of Synthetic Perceptive, Emotive and Cognitive Systems, Department of Technology, Center of Autonomous Systems and Neurorobotics, Universitat Pompeu Fabra Barcelona, Spain ; Institució Catalana de Recerca i Estudis Avançats Barcelona, Spain
| | - Niels Birbaumer
- Institut für Medizinische Psychologie und Verhaltensneurobiologie, Universität Tübingen Tübingen, Germany ; Fondazione Ospedale San Camillo, Istituto di Ricovero e Cura a Carattere Scientifico Venezia, Italy
| | - Andrea Caria
- Institut für Medizinische Psychologie und Verhaltensneurobiologie, Universität Tübingen Tübingen, Germany ; Fondazione Ospedale San Camillo, Istituto di Ricovero e Cura a Carattere Scientifico Venezia, Italy
| |
Collapse
|
27
|
Toppi J, Risetti M, Quitadamo LR, Petti M, Bianchi L, Salinari S, Babiloni F, Cincotti F, Mattia D, Astolfi L. Investigating the effects of a sensorimotor rhythm-based BCI training on the cortical activity elicited by mental imagery. J Neural Eng 2014; 11:035010. [DOI: 10.1088/1741-2560/11/3/035010] [Citation(s) in RCA: 23] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
28
|
Tomita Y, Vialatte FB, Dreyfus G, Mitsukura Y, Bakardjian H, Cichocki A. Bimodal BCI Using Simultaneously NIRS and EEG. IEEE Trans Biomed Eng 2014; 61:1274-84. [DOI: 10.1109/tbme.2014.2300492] [Citation(s) in RCA: 75] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
29
|
Akcakaya M, Peters B, Moghadamfalahi M, Mooney AR, Orhan U, Oken B, Erdogmus D, Fried-Oken M. Noninvasive brain-computer interfaces for augmentative and alternative communication. IEEE Rev Biomed Eng 2014; 7:31-49. [PMID: 24802700 PMCID: PMC6525622 DOI: 10.1109/rbme.2013.2295097] [Citation(s) in RCA: 66] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/08/2023]
Abstract
Brain-computer interfaces (BCIs) promise to provide a novel access channel for assistive technologies, including augmentative and alternative communication (AAC) systems, to people with severe speech and physical impairments (SSPI). Research on the subject has been accelerating significantly in the last decade and the research community took great strides toward making BCI-AAC a practical reality to individuals with SSPI. Nevertheless, the end goal has still not been reached and there is much work to be done to produce real-world-worthy systems that can be comfortably, conveniently, and reliably used by individuals with SSPI with help from their families and care givers who will need to maintain, setup, and debug the systems at home. This paper reviews reports in the BCI field that aim at AAC as the application domain with a consideration on both technical and clinical aspects.
Collapse
|
30
|
Whatever works: a systematic user-centered training protocol to optimize brain-computer interfacing individually. PLoS One 2013; 8:e76214. [PMID: 24086710 PMCID: PMC3781165 DOI: 10.1371/journal.pone.0076214] [Citation(s) in RCA: 47] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2013] [Accepted: 08/28/2013] [Indexed: 11/19/2022] Open
Abstract
This study implemented a systematic user-centered training protocol for a 4-class brain-computer interface (BCI). The goal was to optimize the BCI individually in order to achieve high performance within few sessions for all users. Eight able-bodied volunteers, who were initially naïve to the use of a BCI, participated in 10 sessions over a period of about 5 weeks. In an initial screening session, users were asked to perform the following seven mental tasks while multi-channel EEG was recorded: mental rotation, word association, auditory imagery, mental subtraction, spatial navigation, motor imagery of the left hand and motor imagery of both feet. Out of these seven mental tasks, the best 4-class combination as well as most reactive frequency band (between 8-30 Hz) was selected individually for online control. Classification was based on common spatial patterns and Fisher's linear discriminant analysis. The number and time of classifier updates varied individually. Selection speed was increased by reducing trial length. To minimize differences in brain activity between sessions with and without feedback, sham feedback was provided in the screening and calibration runs in which usually no real-time feedback is shown. Selected task combinations and frequency ranges differed between users. The tasks that were included in the 4-class combination most often were (1) motor imagery of the left hand (2), one brain-teaser task (word association or mental subtraction) (3), mental rotation task and (4) one more dynamic imagery task (auditory imagery, spatial navigation, imagery of the feet). Participants achieved mean performances over sessions of 44-84% and peak performances in single-sessions of 58-93% in this user-centered 4-class BCI protocol. This protocol is highly adjustable to individual users and thus could increase the percentage of users who can gain and maintain BCI control. A high priority for future work is to examine this protocol with severely disabled users.
Collapse
|
31
|
Lotte F, Larrue F, Mühl C. Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design. Front Hum Neurosci 2013; 7:568. [PMID: 24062669 PMCID: PMC3775130 DOI: 10.3389/fnhum.2013.00568] [Citation(s) in RCA: 125] [Impact Index Per Article: 11.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2013] [Accepted: 08/26/2013] [Indexed: 11/13/2022] Open
Abstract
While recent research on Brain-Computer Interfaces (BCI) has highlighted their potential for many applications, they remain barely used outside laboratories. The main reason is their lack of robustness. Indeed, with current BCI, mental state recognition is usually slow and often incorrect. Spontaneous BCI (i.e., mental imagery-based BCI) often rely on mutual learning efforts by the user and the machine, with BCI users learning to produce stable ElectroEncephaloGraphy (EEG) patterns (spontaneous BCI control being widely acknowledged as a skill) while the computer learns to automatically recognize these EEG patterns, using signal processing. Most research so far was focused on signal processing, mostly neglecting the human in the loop. However, how well the user masters the BCI skill is also a key element explaining BCI robustness. Indeed, if the user is not able to produce stable and distinct EEG patterns, then no signal processing algorithm would be able to recognize them. Unfortunately, despite the importance of BCI training protocols, they have been scarcely studied so far, and used mostly unchanged for years. In this paper, we advocate that current human training approaches for spontaneous BCI are most likely inappropriate. We notably study instructional design literature in order to identify the key requirements and guidelines for a successful training procedure that promotes a good and efficient skill learning. This literature study highlights that current spontaneous BCI user training procedures satisfy very few of these requirements and hence are likely to be suboptimal. We therefore identify the flaws in BCI training protocols according to instructional design principles, at several levels: in the instructions provided to the user, in the tasks he/she has to perform, and in the feedback provided. For each level, we propose new research directions that are theoretically expected to address some of these flaws and to help users learn the BCI skill more efficiently.
Collapse
|
32
|
On the control of brain-computer interfaces by users with cerebral palsy. Clin Neurophysiol 2013; 124:1787-97. [PMID: 23684128 DOI: 10.1016/j.clinph.2013.02.118] [Citation(s) in RCA: 71] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2012] [Revised: 02/27/2013] [Accepted: 02/28/2013] [Indexed: 11/23/2022]
|
33
|
Moghimi S, Kushki A, Guerguerian AM, Chau T. A review of EEG-based brain-computer interfaces as access pathways for individuals with severe disabilities. Assist Technol 2013; 25:99-110. [PMID: 23923692 DOI: 10.1080/10400435.2012.723298] [Citation(s) in RCA: 67] [Impact Index Per Article: 6.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022] Open
Abstract
Electroencephalography (EEG) is a non-invasive method for measuring brain activity and is a strong candidate for brain-computer interface (BCI) development. While BCIs can be used as a means of communication for individuals with severe disabilities, the majority of existing studies have reported BCI evaluations by able-bodied individuals. Considering the many differences in body functions and usage scenarios between individuals with disabilities and able-bodied individuals, involvement of the target population in BCI evaluation is necessary. In this review, 39 studies reporting EEG-oriented BCI assessment by individuals with disabilities were identified in the past decade. With respect to participant populations, a need for assessing BCI performance for the pediatric population with severe disabilities was identified as an important future direction. Acquiring a reliable communication pathway during early stages of development is crucial in avoiding learned helplessness in pediatric-onset disabilities. With respect to evaluation, augmenting traditional measures of system performance with those relating to contextual factors was recommended for realizing user-centered designs appropriate for integration in real-life. Considering indicators of user state and developing more effective training paradigms are recommended for future studies of BCI involving individuals with disabilities.
Collapse
Affiliation(s)
- Saba Moghimi
- Institute of Biomaterials and Biomedical Engineering, University of Toronto, Toronto, Ontario, Canada
| | | | | | | |
Collapse
|
34
|
Tonin L, Leeb R, Sobolewski A, Millán JDR. An online EEG BCI based on covert visuospatial attention in absence of exogenous stimulation. J Neural Eng 2013; 10:056007. [DOI: 10.1088/1741-2560/10/5/056007] [Citation(s) in RCA: 37] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
|
35
|
Arvaneh M, Guan C, Ang KK, Quek C. Optimizing spatial filters by minimizing within-class dissimilarities in electroencephalogram-based brain-computer interface. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2013; 24:610-619. [PMID: 24808381 DOI: 10.1109/tnnls.2013.2239310] [Citation(s) in RCA: 58] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
A major challenge in electroencephalogram (EEG)-based brain-computer interfaces (BCIs) is the inherent nonstationarities in the EEG data. Variations of the signal properties from intra and inter sessions often lead to deteriorated BCI performances, as features extracted by methods such as common spatial patterns (CSP) are not invariant against the changes. To extract features that are robust and invariant, this paper proposes a novel spatial filtering algorithm called Kullback-Leibler (KL) CSP. The CSP algorithm only considers the discrimination between the means of the classes, but does not consider within-class scatters information. In contrast, the proposed KLCSP algorithm simultaneously maximizes the discrimination between the class means, and minimizes the within-class dissimilarities measured by a loss function based on the KL divergence. The performance of the proposed KLCSP algorithm is compared against two existing algorithms, CSP and stationary CSP (sCSP), using the publicly available BCI competition III dataset IVa and a large dataset from stroke patients performing neuro-rehabilitation. The results show that the proposed KLCSP algorithm significantly outperforms both the CSP and the sCSP algorithms, in terms of classification accuracy, by reducing within-class variations. This results in more compact and separable features.
Collapse
|
36
|
Friedrich EVC, Scherer R, Neuper C. Long-term evaluation of a 4-class imagery-based brain-computer interface. Clin Neurophysiol 2013; 124:916-27. [PMID: 23290926 DOI: 10.1016/j.clinph.2012.11.010] [Citation(s) in RCA: 31] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2012] [Revised: 09/24/2012] [Accepted: 11/17/2012] [Indexed: 10/27/2022]
Abstract
OBJECTIVE The study aimed to improve brain-computer interface (BCI)-usability by using distinct control strategies and evaluating performance, brain activity and psychological variables on a long-term basis over several months. METHODS Fourteen able-bodied users participated in 10 sessions, plus a follow-up session. Users were trained to control an EEG-based 4-class BCI with the mental tasks, word association, mental subtraction, spatial navigation, and motor imagery. RESULTS Eight users reached mean accuracies of 61-72% and managed to control all 4 classes above chance in single-sessions. Performance and brain patterns stayed stable over 10 weeks without training. Motor imagery showed the best performance and most distinct brain patterns. Participants' fear of incompetence decreased while the quality of their imagery and task ease increased over sessions. The evaluation of feedback differed between tasks and correlated with performance. CONCLUSION Users can control a real-time 4-class BCI, driven by distinct mental tasks, with stable performance over months. However, general performance was rather low for effective BCI control in daily life. Possibilities for future optimizations to increase performance are discussed. SIGNIFICANCE The evaluation of alternatives to motor imagery, long-term BCI use, and psychological variables is important to improve usability for mental imagery-based BCIs.
Collapse
|
37
|
Iturrate I, Montesano L, Chavarriaga R, del R Millán J, Minguez J. Minimizing calibration time using inter-subject information of single-trial recognition of error potentials in brain-computer interfaces. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2012; 2011:6369-72. [PMID: 22255795 DOI: 10.1109/iembs.2011.6091572] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
One of the main problems of both synchronous and asynchronous EEG-based BCIs is the need of an initial calibration phase before the system can be used. This phase is necessary due to the high non-stationarity of the EEG, since it changes between sessions and users. The calibration process limits the BCI systems to scenarios where the outputs are very controlled, and makes these systems non-friendly and exhausting for the users. Although it has been studied how to reduce calibration time for asynchronous signals, it is still an open issue for event-related potentials. Here, we propose the minimization of the calibration time on single-trial error potentials by using classifiers based on inter-subject information. The results show that it is possible to have a classifier with a high performance from the beginning of the experiment, and which is able to adapt itself making the calibration phase shorter and transparent to the user.
Collapse
Affiliation(s)
- Iñaki Iturrate
- Instituto de Investigación en Ingenieráía de Aragón and Dpto de Informáticae Ingeniería de Sistemas, Universidad de Zaragoza, Spain.
| | | | | | | | | |
Collapse
|
38
|
A frequency-temporal-spatial method for motor-related electroencephalography pattern recognition by comprehensive feature optimization. Comput Biol Med 2012; 42:353-63. [PMID: 22348825 DOI: 10.1016/j.compbiomed.2011.11.014] [Citation(s) in RCA: 11] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2010] [Revised: 10/19/2011] [Accepted: 11/28/2011] [Indexed: 11/22/2022]
Abstract
Either imagined or actual movements lead to a combination of electroencephalography signals with distinctive frequency, temporal and spatial characteristics, which correspond to various motor-related neural activities. This frequency-temporal-spatial pattern is the key of motor intention decoding which is the basis of brain-computer interfaces by motor imagery. We present a new method for motor-related electroencephalography recognition which comprehensively optimizes the frequency-time-space features in a user-specific way. The recognition work focuses on three points: proper time and frequency domain segmentation, spatial optimization based on common spatial pattern filters and feature importance evaluation. We show that by combining the advantages of these optimizational methods, the proposed algorithm effectively improves motor task classification, and the recognized signal chanracteristics can be used to visualize the motor related electroencephalography patterns under different conditions.
Collapse
|
39
|
Brain computer interfaces, a review. SENSORS 2012; 12:1211-79. [PMID: 22438708 PMCID: PMC3304110 DOI: 10.3390/s120201211] [Citation(s) in RCA: 716] [Impact Index Per Article: 59.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/29/2011] [Revised: 01/16/2012] [Accepted: 01/29/2012] [Indexed: 11/16/2022]
Abstract
A brain-computer interface (BCI) is a hardware and software communications system that permits cerebral activity alone to control computers or external devices. The immediate goal of BCI research is to provide communications capabilities to severely disabled people who are totally paralyzed or 'locked in' by neurological neuromuscular disorders, such as amyotrophic lateral sclerosis, brain stem stroke, or spinal cord injury. Here, we review the state-of-the-art of BCIs, looking at the different steps that form a standard BCI: signal acquisition, preprocessing or signal enhancement, feature extraction, classification and the control interface. We discuss their advantages, drawbacks, and latest advances, and we survey the numerous technologies reported in the scientific literature to design each step of a BCI. First, the review examines the neuroimaging modalities used in the signal acquisition step, each of which monitors a different functional brain activity such as electrical, magnetic or metabolic activity. Second, the review discusses different electrophysiological control signals that determine user intentions, which can be detected in brain activity. Third, the review includes some techniques used in the signal enhancement step to deal with the artifacts in the control signals and improve the performance. Fourth, the review studies some mathematic algorithms used in the feature extraction and classification steps which translate the information in the control signals into commands that operate a computer or other device. Finally, the review provides an overview of various BCI applications that control a range of devices.
Collapse
|
40
|
Friedrich EVC, Scherer R, Neuper C. The effect of distinct mental strategies on classification performance for brain-computer interfaces. Int J Psychophysiol 2012; 84:86-94. [PMID: 22289414 DOI: 10.1016/j.ijpsycho.2012.01.014] [Citation(s) in RCA: 66] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2011] [Revised: 12/22/2011] [Accepted: 01/12/2012] [Indexed: 01/09/2023]
Abstract
Motor imagery is the task most commonly used to induce changes in electroencephalographic (EEG) signals for mental imagery-based brain computer interfacing (BCI). In this study, we investigated EEG patterns that were induced by seven different mental tasks (i.e. mental rotation, word association, auditory imagery, mental subtraction, spatial navigation, imagery of familiar faces and motor imagery) and evaluated the binary classification performance. The aim was to provide a broad range of reliable and user-appropriate tasks to make individual optimization of BCI control strategies possible. Nine users participated in four sessions of multi-channel EEG recordings. Mental tasks resulting most frequently in good binary classification performance include mental subtraction, word association, motor imagery and mental rotation. Our results indicate that a combination of 'brain-teasers' - tasks that require problem specific mental work (e.g. mental subtraction, word association) - and dynamic imagery tasks (e.g. motor imagery) result in highly distinguishable brain patterns that lead to an increased performance.
Collapse
|
41
|
Billinger M, Daly I, Kaiser V, Jin J, Allison BZ, Müller-Putz GR, Brunner C. Is It Significant? Guidelines for Reporting BCI Performance. TOWARDS PRACTICAL BRAIN-COMPUTER INTERFACES 2012. [DOI: 10.1007/978-3-642-29746-5_17] [Citation(s) in RCA: 39] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
|
42
|
|
43
|
Besserve M, Martinerie J, Garnero L. Improving quantification of functional networks with EEG inverse problem: Evidence from a decoding point of view. Neuroimage 2011; 55:1536-47. [DOI: 10.1016/j.neuroimage.2011.01.056] [Citation(s) in RCA: 31] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2010] [Revised: 12/20/2010] [Accepted: 01/20/2011] [Indexed: 11/16/2022] Open
|
44
|
Millán JDR, Rupp R, Müller-Putz GR, Murray-Smith R, Giugliemma C, Tangermann M, Vidaurre C, Cincotti F, Kübler A, Leeb R, Neuper C, Müller KR, Mattia D. Combining Brain-Computer Interfaces and Assistive Technologies: State-of-the-Art and Challenges. Front Neurosci 2010; 4. [PMID: 20877434 PMCID: PMC2944670 DOI: 10.3389/fnins.2010.00161] [Citation(s) in RCA: 245] [Impact Index Per Article: 17.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2010] [Accepted: 08/01/2010] [Indexed: 11/29/2022] Open
Abstract
In recent years, new research has brought the field of electroencephalogram (EEG)-based brain–computer interfacing (BCI) out of its infancy and into a phase of relative maturity through many demonstrated prototypes such as brain-controlled wheelchairs, keyboards, and computer games. With this proof-of-concept phase in the past, the time is now ripe to focus on the development of practical BCI technologies that can be brought out of the lab and into real-world applications. In particular, we focus on the prospect of improving the lives of countless disabled individuals through a combination of BCI technology with existing assistive technologies (AT). In pursuit of more practical BCIs for use outside of the lab, in this paper, we identify four application areas where disabled individuals could greatly benefit from advancements in BCI technology, namely, “Communication and Control”, “Motor Substitution”, “Entertainment”, and “Motor Recovery”. We review the current state of the art and possible future developments, while discussing the main research issues in these four areas. In particular, we expect the most progress in the development of technologies such as hybrid BCI architectures, user–machine adaptation algorithms, the exploitation of users’ mental states for BCI reliability and confidence measures, the incorporation of principles in human–computer interaction (HCI) to improve BCI usability, and the development of novel BCI technology including better EEG devices.
Collapse
Affiliation(s)
- J D R Millán
- Defitech Chair in Non-Invasive Brain-Machine Interface, Center for Neuroprosthetics, School of Engineering, Ecole Polytechnique Fédérale de Lausanne Lausanne, Switzerland
| | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
45
|
Xu Q, Zhou H, Wang Y, Huang J. Fuzzy support vector machine for classification of EEG signals using wavelet-based features. Med Eng Phys 2009; 31:858-65. [DOI: 10.1016/j.medengphy.2009.04.005] [Citation(s) in RCA: 105] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2008] [Revised: 04/10/2009] [Accepted: 04/28/2009] [Indexed: 11/29/2022]
|
46
|
Some computational aspects of the brain computer interfaces based on inner music. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2009:950403. [PMID: 19503802 PMCID: PMC2688653 DOI: 10.1155/2009/950403] [Citation(s) in RCA: 12] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/21/2008] [Revised: 01/11/2009] [Accepted: 03/11/2009] [Indexed: 12/03/2022]
Abstract
We discuss the BCI based on inner tones and inner music. We had some success in the detection of inner tones, the imagined tones which are not sung aloud. Rather easily imagined and controlled, they offer a set of states usable for BCI, with high information capacity and high transfer rates. Imagination of sounds or musical tunes could provide a multicommand language for BCI, as if using the natural language. Moreover, this approach could be used to test musical abilities. Such BCI interface could be superior when there is a need for a broader command language. Some computational
estimates and unresolved difficulties are presented.
Collapse
|
47
|
Zhou J, Yao J, Deng J, Dewald JPA. EEG-based classification for elbow versus shoulder torque intentions involving stroke subjects. Comput Biol Med 2009; 39:443-52. [PMID: 19380125 DOI: 10.1016/j.compbiomed.2009.02.004] [Citation(s) in RCA: 38] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2007] [Revised: 02/17/2009] [Accepted: 02/26/2009] [Indexed: 11/26/2022]
Abstract
The ultimate aim for classifying elbow versus shoulder torque intentions is to develop robust brain-computer interface (BCI) devices for patients who suffer from movement disorders following brain injury such as stroke. In this paper, we investigate the advanced classification approach classifier-enhanced time-frequency synthesized spatial pattern algorithm (classifier-enhanced TFSP) in classifying a subject's intent of generating an isometric shoulder abduction (SABD) or elbow flexion (EF) torque using signals obtained from 163 scalp electroencephalographic (EEG) electrodes. Two classifiers, the support vector classifier (SVC) and the classification and regression tree (CART), are integrated in the TFSP algorithm that decomposes the signal into a weighted time, frequency and spatial feature space. The resulting high-performing methods (SVC-TFSP and CART-TFSP) are then applied to experimental data collected in four healthy subjects and two stroke subjects. Results are compared with the original TFSP, and significantly higher reliability in both healthy subjects (92% averaged over four healthy subjects) and stroke subjects (75% averaged over two subjects) are achieved. The accuracies of classifier-enhanced TFSP methods are further improved after a rejection scheme is applied (approximately 100% in healthy subjects and >80% in stroke subjects). The results are among the highest reliability reported in literature for tasks with spatial representations on the motor cortex as close as shoulder and elbow. The paper also discusses the impact of applying rejection strategy in detail and reports the existence of an optimal rejection rate on a stroke subject. The results indicate that the proposed algorithms are promising for future use of rehabilitative BCI applications in neurologically impaired patients.
Collapse
Affiliation(s)
- Jie Zhou
- Department of Computer Science, Northern Illinois University, USA.
| | | | | | | |
Collapse
|
48
|
Liu D, Pang Z, Lloyd SR. A neural network method for detection of obstructive sleep apnea and narcolepsy based on pupil size and EEG. ACTA ACUST UNITED AC 2008; 19:308-18. [PMID: 18269961 DOI: 10.1109/tnn.2007.908634] [Citation(s) in RCA: 35] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Electroencephalogram (EEG) is able to indicate states of mental activity ranging from concentrated cognitive efforts to sleepiness. Such mental activity can be reflected by EEG energy. In particular, intrusion of EEG theta wave activity into the beta activity of active wakefulness has been interpreted as ensuing sleepiness. Pupil behavior can also provide information regarding alertness. This paper develops an innovative signal classification method that is capable of differentiating subjects with sleep disorders which cause excessive daytime sleepiness (EDS) from normal control subjects who do not have a sleep disorder based on EEG and pupil size. Subjects with sleep disorders include persons with untreated obstructive sleep apnea (OSA) and narcolepsy. The Yoss pupil staging rule is used to scale levels of wakefulness and at the same time theta energy ratios are calculated from the same 2-s sliding windows by Fourier or wavelet transforms. Then, an artificial neural network (NN) of modified adaptive resonance theory (ART2) is utilized to identify the two groups within a combined group of subjects including those with OSA and healthy controls. This grouping from the NN is then compared with the actual diagnostic classification of subjects as OSA or controls and is found to be 91% accurate in differentiating between the two groups. The same algorithm results in 90% correct differentiation between narcoleptic and control subjects.
Collapse
Affiliation(s)
- D Liu
- Department of Electrical and Computer Engineering, University of Illinois at Chicago, Chicago, IL 60607-7053, USA.
| | | | | |
Collapse
|
49
|
Blankertz B, Dornhege G, Krauledat M, Müller KR, Curio G. The non-invasive Berlin Brain–Computer Interface: Fast acquisition of effective performance in untrained subjects. Neuroimage 2007; 37:539-50. [PMID: 17475513 DOI: 10.1016/j.neuroimage.2007.01.051] [Citation(s) in RCA: 390] [Impact Index Per Article: 22.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2005] [Revised: 01/16/2007] [Accepted: 01/18/2007] [Indexed: 11/29/2022] Open
Abstract
Brain-Computer Interface (BCI) systems establish a direct communication channel from the brain to an output device. These systems use brain signals recorded from the scalp, the surface of the cortex, or from inside the brain to enable users to control a variety of applications. BCI systems that bypass conventional motor output pathways of nerves and muscles can provide novel control options for paralyzed patients. One classical approach to establish EEG-based control is to set up a system that is controlled by a specific EEG feature which is known to be susceptible to conditioning and to let the subjects learn the voluntary control of that feature. In contrast, the Berlin Brain-Computer Interface (BBCI) uses well established motor competencies of its users and a machine learning approach to extract subject-specific patterns from high-dimensional features optimized for detecting the user's intent. Thus the long subject training is replaced by a short calibration measurement (20 min) and machine learning (1 min). We report results from a study in which 10 subjects, who had no or little experience with BCI feedback, controlled computer applications by voluntary imagination of limb movements: these intentions led to modulations of spontaneous brain activity specifically, somatotopically matched sensorimotor 7-30 Hz rhythms were diminished over pericentral cortices. The peak information transfer rate was above 35 bits per minute (bpm) for 3 subjects, above 23 bpm for two, and above 12 bpm for 3 subjects, while one subject could achieve no BCI control. Compared to other BCI systems which need longer subject training to achieve comparable results, we propose that the key to quick efficiency in the BBCI system is its flexibility due to complex but physiologically meaningful features and its adaptivity which respects the enormous inter-subject variability.
Collapse
|
50
|
Cincotti F, Mattia D, Aloise F, Bufalari S, Astolfi L, De Vico Fallani F, Tocci A, Bianchi L, Marciani MG, Gao S, Millan J, Babiloni F. High-resolution EEG techniques for brain-computer interface applications. J Neurosci Methods 2007; 167:31-42. [PMID: 17706292 DOI: 10.1016/j.jneumeth.2007.06.031] [Citation(s) in RCA: 55] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2006] [Revised: 06/29/2007] [Accepted: 06/29/2007] [Indexed: 10/23/2022]
Abstract
High-resolution electroencephalographic (HREEG) techniques allow estimation of cortical activity based on non-invasive scalp potential measurements, using appropriate models of volume conduction and of neuroelectrical sources. In this study we propose an application of this body of technologies, originally developed to obtain functional images of the brain's electrical activity, in the context of brain-computer interfaces (BCI). Our working hypothesis predicted that, since HREEG pre-processing removes spatial correlation introduced by current conduction in the head structures, by providing the BCI with waveforms that are mostly due to the unmixed activity of a small cortical region, a more reliable classification would be obtained, at least when the activity to detect has a limited generator, which is the case in motor related tasks. HREEG techniques employed in this study rely on (i) individual head models derived from anatomical magnetic resonance images, (ii) distributed source model, composed of a layer of current dipoles, geometrically constrained to the cortical mantle, (iii) depth-weighted minimum L(2)-norm constraint and Tikhonov regularization for linear inverse problem solution and (iv) estimation of electrical activity in cortical regions of interest corresponding to relevant Brodmann areas. Six subjects were trained to learn self modulation of sensorimotor EEG rhythms, related to the imagination of limb movements. Off-line EEG data was used to estimate waveforms of cortical activity (cortical current density, CCD) on selected regions of interest. CCD waveforms were fed into the BCI computational pipeline as an alternative to raw EEG signals; spectral features are evaluated through statistical tests (r(2) analysis), to quantify their reliability for BCI control. These results are compared, within subjects, to analogous results obtained without HREEG techniques. The processing procedure was designed in such a way that computations could be split into a setup phase (which includes most of the computational burden) and the actual EEG processing phase, which was limited to a single matrix multiplication. This separation allowed to make the procedure suitable for on-line utilization, and a pilot experiment was performed. Results show that lateralization of electrical activity, which is expected to be contralateral to the imagined movement, is more evident on the estimated CCDs than in the scalp potentials. CCDs produce a pattern of relevant spectral features that is more spatially focused, and has a higher statistical significance (EEG: 0.20+/-0.114 S.D.; CCD: 0.55+/-0.16 S.D.; p=10(-5)). A pilot experiment showed that a trained subject could utilize voluntary modulation of estimated CCDs for accurate (eight targets) on-line control of a cursor. This study showed that it is practically feasible to utilize HREEG techniques for on-line operation of a BCI system; off-line analysis suggests that accuracy of BCI control is enhanced by the proposed method.
Collapse
|