1
|
Pan J, Chen X, Ban N, He J, Chen J, Huang H. Advances in P300 brain-computer interface spellers: toward paradigm design and performance evaluation. Front Hum Neurosci 2022; 16:1077717. [PMID: 36618996 PMCID: PMC9810759 DOI: 10.3389/fnhum.2022.1077717] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2022] [Accepted: 11/23/2022] [Indexed: 12/24/2022] Open
Abstract
A brain-computer interface (BCI) is a non-muscular communication technology that provides an information exchange channel for our brains and external devices. During the decades, BCI has made noticeable progress and has been applied in many fields. One of the most traditional BCI applications is the BCI speller. This article primarily discusses the progress of research into P300 BCI spellers and reviews four types of P300 spellers: single-modal P300 spellers, P300 spellers based on multiple brain patterns, P300 spellers with multisensory stimuli, and P300 spellers with multiple intelligent techniques. For each type of P300 speller, we further review several representative P300 spellers, including their design principles, paradigms, algorithms, experimental performance, and corresponding advantages. We particularly emphasized the paradigm design ideas, including the overall layout, individual symbol shapes and stimulus forms. Furthermore, several important issues and research guidance for the P300 speller were identified. We hope that this review can assist researchers in learning the new ideas of these novel P300 spellers and enhance their practical application capability.
Collapse
Affiliation(s)
- Jiahui Pan
- *Correspondence: Jiahui Pan Haiyun Huang
| | | | | | | | | | | |
Collapse
|
2
|
Huggins JE, Karlsson P, Warschausky SA. Challenges of brain-computer interface facilitated cognitive assessment for children with cerebral palsy. Front Hum Neurosci 2022; 16:977042. [PMID: 36204719 PMCID: PMC9530314 DOI: 10.3389/fnhum.2022.977042] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2022] [Accepted: 08/23/2022] [Indexed: 11/13/2022] Open
Abstract
Brain-computer interfaces (BCIs) have been successfully used by adults, but little information is available on BCI use by children, especially children with severe multiple impairments who may need technology to facilitate communication. Here we discuss the challenges of using non-invasive BCI with children, especially children who do not have another established method of communication with unfamiliar partners. Strategies to manage these challenges require consideration of multiple factors related to accessibility, cognition, and participation. These factors include decisions regarding where (home, clinic, or lab) participation will take place, the number of sessions involved, and the degree of participation necessary for success. A strategic approach to addressing the unique challenges inherent in BCI use by children with disabilities will increase the potential for successful BCI calibration and adoption of BCI as a valuable access method for children with the most significant impairments in movement and communication.
Collapse
Affiliation(s)
- Jane E. Huggins
- Direct Brain Interface Laboratory, Department of Physical Medicine and Rehabilitation, Neuroscience Graduate Program, University of Michigan, Ann Arbor, MI, United States
- Direct Brain Interface Laboratory, Department of Biomedical Engineering, Neuroscience Graduate Program, University of Michigan, Ann Arbor, MI, United States
| | - Petra Karlsson
- Theme Technology, Faculty of Medicine and Health, Cerebral Palsy Alliance, The University of Sydney, Sydney, NSW, Australia
| | - Seth A. Warschausky
- Adaptive Cognitive Assessment Laboratory, Department of Physical Medicine and Rehabilitation, University of Michigan, Ann Arbor, MI, United States
| |
Collapse
|
3
|
Yue Z, Wu Q, Ren SY, Li M, Shi B, Pan Y, Wang J. A novel multiple time-frequency sequential coding strategy for hybrid brain-computer interface. Front Hum Neurosci 2022; 16:859259. [PMID: 35966991 PMCID: PMC9372511 DOI: 10.3389/fnhum.2022.859259] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2022] [Accepted: 06/28/2022] [Indexed: 11/18/2022] Open
Abstract
Background For brain-computer interface (BCI) communication, electroencephalography provides a preferable choice due to its high temporal resolution and portability over other neural recording techniques. However, current BCIs are unable to sufficiently use the information from time and frequency domains simultaneously. Thus, we proposed a novel hybrid time-frequency paradigm to investigate better ways of using the time and frequency information. Method We adopt multiple omitted stimulus potential (OSP) and steady-state motion visual evoked potential (SSMVEP) to design the hybrid paradigm. A series of pre-experiments were undertaken to study factors that would influence the feasibility of the hybrid paradigm and the interaction between multiple features. After that, a novel Multiple Time-Frequencies Sequential Coding (MTFSC) strategy was introduced and explored in experiments. Results Omissions with multiple short and long durations could effectively elicit time and frequency features, including the multi-OSP, ERP, and SSVEP in this hybrid paradigm. The MTFSC was feasible and efficient. The preliminary online analysis showed that the accuracy and the ITR of the nine-target stimulator over thirteen subjects were 89.04% and 36.37 bits/min. Significance This study first combined the SSMVEP and multi-OSP in a hybrid paradigm to produce robust and abundant time features for coding BCI. Meanwhile, the MTFSC proved feasible and showed great potential in improving performance, such as expanding the number of BCI targets by better using time information in specific stimulated frequencies. This study holds promise for designing better BCI systems with a novel coding method.
Collapse
Affiliation(s)
- Zan Yue
- Institute of Robotics and Intelligent Systems, Xi'an Jiaotong University, Xi'an, China
| | - Qiong Wu
- Beijing Tsinghua Changgeng Hospital, Tsinghua University, Beijing, China
| | - Shi-Yuan Ren
- Institute of Robotics and Intelligent Systems, Xi'an Jiaotong University, Xi'an, China
| | - Man Li
- Institute of Robotics and Intelligent Systems, Xi'an Jiaotong University, Xi'an, China
| | - Bin Shi
- Institute of Robotics and Intelligent Systems, Xi'an Jiaotong University, Xi'an, China
| | - Yu Pan
- Beijing Tsinghua Changgeng Hospital, Tsinghua University, Beijing, China
- *Correspondence: Yu Pan
| | - Jing Wang
- Institute of Robotics and Intelligent Systems, Xi'an Jiaotong University, Xi'an, China
- Jing Wang
| |
Collapse
|
4
|
Han J, Liu C, Chu J, Xiao X, Chen L, Xu M, Ming D. Effects of inter-stimulus intervals on concurrent P300 and SSVEP features for hybrid Brain-computer interfaces. J Neurosci Methods 2022; 372:109535. [PMID: 35202615 DOI: 10.1016/j.jneumeth.2022.109535] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2021] [Revised: 01/25/2022] [Accepted: 02/18/2022] [Indexed: 10/19/2022]
Abstract
BACKGROUND Recently, we have implemented a high-speed brain-computer interface (BCI) system with a large instruction set using the concurrent P300 and steady-state visual evoked potential (SSVEP) features (also known as hybrid features). However, it remains unclear how to select inter-stimulus interval (ISI) for the proposed BCI system to balance the encoding efficiency and decoding performance. NEW METHOD This study developed a 6⁎9 hybrid P300-SSVEP BCI system and investigated a series of ISIs ranged from -175ms to 0ms with a step of 25ms. The influence of ISI on the hybrid features was analyzed from several aspects, including the amplitude of the induced features, classification accuracy, information transfer rate (ITR). Twelve naive subjects were recruited for the experiment. RESULTS The results showed the ISI factor had a significant impact on the hybrid features. Specifically, as the values of ISI decreased, the amplitudes of the induced features and accuracies decreased gradually, while the ITRs increased rapidly. It's achieved the highest ITR of 158.50 bits/min when ISI equal to -175ms. COMPARISON WITH EXISTING METHOD The optimal ISI in this study achieved superior performance in comparison with the one we used in the previous study. CONCLUSIONS The ISI can exert an important influence on the P300-SSVEP BCI system and its optimal value is -175ms in this study, which is significant for developing the high-speed BCI system with larger instruction sets in the future.
Collapse
Affiliation(s)
- Jin Han
- Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China
| | - Chuan Liu
- Division of Medicine, Tianjin University, Tianjin 300072, People's Republic of China
| | - Jiayue Chu
- Division of Medicine, Tianjin University, Tianjin 300072, People's Republic of China
| | - Xiaolin Xiao
- Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China; Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China.
| | - Long Chen
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China.
| | - Minpeng Xu
- Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China; Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China
| | - Dong Ming
- Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China; Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China
| |
Collapse
|
5
|
Ma T, Li Y, Huggins JE, Zhu J, Kang J. Bayesian Inferences on Neural Activity in EEG-Based Brain-Computer Interface. J Am Stat Assoc 2022; 117:1122-1133. [PMID: 36313593 PMCID: PMC9609845 DOI: 10.1080/01621459.2022.2041422] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/15/2023]
Abstract
A brain-computer interface (BCI) is a system that translates brain activity into commands to operate technology. A common design for an electroencephalogram (EEG) BCI relies on the classification of the P300 event-related potential (ERP), which is a response elicited by the rare occurrence of target stimuli among common non-target stimuli. Few existing ERP classifiers directly explore the underlying mechanism of the neural activity. To this end, we perform a novel Bayesian analysis of the probability distribution of multi-channel real EEG signals under the P300 ERP-BCI design. We aim to identify relevant spatial temporal differences of the neural activity, which provides statistical evidence of P300 ERP responses and helps design individually efficient and accurate BCIs. As one key finding of our single participant analysis, there is a 90% posterior probability that the target ERPs of the channels around visual cortex reach their negative peaks around 200 milliseconds post-stimulus. Our analysis identifies five important channels (PO7, PO8, Oz, P4, Cz) for the BCI speller leading to a 100% prediction accuracy. From the analyses of nine other participants, we consistently select the identified five channels, and the selection frequencies are robust to small variations of bandpass filters and kernel hyper-parameters.
Collapse
Affiliation(s)
- Tianwen Ma
- Department of Biostatistics, University of Michigan
| | - Yang Li
- Department of Statistics, University of Michigan
| | - Jane E Huggins
- Department of Physical Medicine and Rehabilitation and Department of Biomedical Engineering, University of Michigan
| | - Ji Zhu
- Department of Statistics, University of Michigan
| | - Jian Kang
- Department of Biostatistics, University of Michigan
| |
Collapse
|
6
|
Rathi N, Singla R, Tiwari S. A novel approach for designing authentication system using a picture based P300 speller. Cogn Neurodyn 2021; 15:805-824. [PMID: 34603543 PMCID: PMC8448816 DOI: 10.1007/s11571-021-09664-3] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2020] [Revised: 09/30/2020] [Accepted: 01/08/2021] [Indexed: 10/22/2022] Open
Abstract
Due to great advances in the field of information technology, the need for a more reliable authentication system has been growing rapidly for protecting the individual or organizational assets from online frauds. In the past, many authentication techniques have been proposed like password and tokens but these techniques suffer from many shortcomings such as offline attacks (guessing) and theft. To overcome these shortcomings, in this paper brain signal based authentication system is proposed. A Brain-Computer Interface (BCI) is a tool that provides direct human-computer interaction by analyzing brain signals. In this study, a person authentication approach that can effectively recognize users by generating unique brain signal features in response to pictures of different objects is presented. This study focuses on a P300 BCI for authentication system design. Also, three classifiers were tested: Quadratic Discriminant Analysis (QDA), K-Nearest Neighbor, and Quadratic Support Vector Machine. The results showed that the proposed visual stimuli with pictures as selection attributes obtained significantly higher classification accuracies (97%) and information transfer rates (37.14 bits/min) as compared to the conventional paradigm. The best performance was observed with the QDA as compare to other classifiers. This method is advantageous for developing brain signal based authentication application as it cannot be forged (like Shoulder surfing) and can still be used for disabled users with a brain in good running condition. The results show that reduced matrix size and modified visual stimulus typically affects the accuracy and communication speed of P300 BCI performance.
Collapse
Affiliation(s)
- Nikhil Rathi
- ICE Department, Dr. B. R. Ambedkar NIT Jalandhar, GT Road Bye-Pass, Jalandhar, Punjab 144011 India
| | - Rajesh Singla
- ICE Department, Dr. B. R. Ambedkar NIT Jalandhar, GT Road Bye-Pass, Jalandhar, Punjab 144011 India
| | - Sheela Tiwari
- ICE Department, Dr. B. R. Ambedkar NIT Jalandhar, GT Road Bye-Pass, Jalandhar, Punjab 144011 India
| |
Collapse
|
7
|
Sutaj N, Walchshofer M, Schreiner L, Turchet L, Pretl H, Guger C. Evaluating a Novel P300-Based Real-Time Image Ranking BCI. FRONTIERS IN COMPUTER SCIENCE 2021. [DOI: 10.3389/fcomp.2021.661224] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Abstract
Brain–computer interfaces (BCIs) establish communication between a human brain and a computer or external devices by translating the electroencephalography (EEG) signal into computer commands. After stimulating a sensory organ, a positive deflection of the EEG signal between 250 and 700 ms can be measured. This signal component of the event-related potential (ERP) is called “P300.” Numerous studies have provided evidence that the P300 amplitude and latency are linked to sensory perception, engagement, and cognition. Combining the advances in technology, classification methods, and signal processing, we developed a novel image ranking system called the Unicorn Blondy Check. In this study, the application was tested on 21 subjects using three different visual oddball paradigms. Two consisted of female faces and gray-scale images, while the third test paradigm consisted of familiar and unfamiliar faces. The images were displayed for a duration of 150 ms in a randomized order. The system was trained using 50 trials and tested with 30 trials. The EEG data were acquired using the Unicorn Hybrid Black eight-channel BCI system. These synchronized recordings were analyzed, and the achieved classification accuracies were calculated. The EEG signal was averaged over all participants and for every paradigm separately. Analysis of the EEG data revealed a significant shift in the P300 latency dependent on the paradigm and decreased amplitude for a lower target to non-target ratio. The image ranking application achieved a mean accuracy of 100 and 95.5% for ranking female faces above gray-scale images with ratios of 1:11 and 5:11, respectively. In the case of four familiar faces to 24 unfamiliar faces, 86.4% was reached. The obtained results illustrate this novel system’s functionality due to accuracies above chance levels for all subjects.
Collapse
|
8
|
Yu X, da Silva-Sauer L, Donchin E. Habituation of P300 in the Use of P300-based Brain-Computer Interface Spellers: Individuals With Amyotrophic Lateral Sclerosis Versus Age-Matched Controls. Clin EEG Neurosci 2021; 52:221-230. [PMID: 32419492 DOI: 10.1177/1550059420918755] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
The P300-based brain-computer interface speller can provide motor independent communication to individuals with amyotrophic lateral sclerosis (ALS), a progressive neurodegenerative disorder that affects the motor system. P300 amplitude stability is critical for operation of the P300 speller. The P300 has good long-term stability, but to our knowledge, short-term habituation in the P300 speller has not been studied. In the current study, 15 participants: 8 ALS patients and 7 age-matched healthy volunteers (HVs), used 2 versions of P300 spellers, Face speller and Flash speller, each for 30 minutes. The ALS group performed as well as the HVs in both spellers and HVs did better with the Face speller than Flash speller while the ALS group performed equally well in both spellers. Neither intra-run P300 habituation nor inter-run P300 habituation was found. The P300 speller could be a reliable communication device for individuals with ALS.
Collapse
Affiliation(s)
- Xiaoqian Yu
- Department of Psychology, 7831University of South Florida, Tampa, FL, USA
| | - Leandro da Silva-Sauer
- Department of Psychology, 7831University of South Florida, Tampa, FL, USA.,123204Federal University of Paraiba, João Pessoa, Paraiba, Brazil
| | - Emanuel Donchin
- Department of Psychology, 7831University of South Florida, Tampa, FL, USA
| |
Collapse
|
9
|
Shukla PK, Chaurasiya RK, Verma S. Performance improvement of P300-based home appliances control classification using convolution neural network. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2020.102220] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
10
|
Rathi N, Singla R, Tiwari S. Authentication framework for security application developed using a pictorial P300 speller. BRAIN-COMPUTER INTERFACES 2020. [DOI: 10.1080/2326263x.2020.1860520] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Affiliation(s)
- Nikhil Rathi
- ICE Department, Dr. B. R. Ambedkar NIT Jalandhar , Jalandhar, Punjab, India
| | - Rajesh Singla
- ICE Department, Dr. B. R. Ambedkar NIT Jalandhar , Jalandhar, Punjab, India
| | - Sheela Tiwari
- ICE Department, Dr. B. R. Ambedkar NIT Jalandhar , Jalandhar, Punjab, India
| |
Collapse
|
11
|
Kshirsagar GB, Londhe ND. Weighted Ensemble of Deep Convolution Neural Networks for Single-Trial Character Detection in Devanagari-Script-Based P300 Speller. IEEE Trans Cogn Dev Syst 2020. [DOI: 10.1109/tcds.2019.2942437] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
12
|
Fernández-Rodríguez Á, Medina-Juliá MT, Velasco-Álvarez F, Ron-Angevin R. Effects of Spatial Stimulus Overlap in a Visual P300-based Brain-computer Interface. Neuroscience 2020; 431:134-142. [PMID: 32081721 DOI: 10.1016/j.neuroscience.2020.02.011] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2019] [Revised: 02/07/2020] [Accepted: 02/10/2020] [Indexed: 11/19/2022]
Abstract
The rapid serial visual presentation (RSVP) paradigm seems to be one of the most appropriate for patients using P300-based brain-computer interface (BCI) applications, since non-ocular movements are required. However, according to previous works, the use of different locations for each stimulus may improve performance. Thus, the aim of the present work is to explore how spatial overlap between stimuli influences performance in controlling a visual P300-based BCI. Nineteen participants were tested using four levels of overlap between two stimuli: 100%, 66.7%, 33.3% and 0%. Significant differences in accuracy were found between the 0% overlapped condition and all the other conditions, and between 33.3% and higher overlap (66.7% and 100%). These results can be explained due to a modulation in the non-target stimulus amplitude signal caused by the overlapping factor. In short, the stimulus overlap provokes a modulation in performance using a P300-based BCI; this should be considered in future BCI proposals in which an optimal surface exploitation is convenient and potential users have only residual ocular movement.
Collapse
Affiliation(s)
| | | | | | - Ricardo Ron-Angevin
- Departamento de Tecnología Electrónica, Universidad de Málaga, 29071 Malaga, Spain
| |
Collapse
|
13
|
Li S, Jin J, Daly I, Zuo C, Wang X, Cichocki A. Comparison of the ERP-Based BCI Performance Among Chromatic (RGB) Semitransparent Face Patterns. Front Neurosci 2020; 14:54. [PMID: 32082118 PMCID: PMC7006297 DOI: 10.3389/fnins.2020.00054] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2019] [Accepted: 01/14/2020] [Indexed: 11/18/2022] Open
Abstract
Objective Previous studies have shown that combing with color properties may be used as part of the display presented to BCI users in order to improve performance. Build on this, we explored the effects of combinations of face stimuli with three primary colors (RGB) on BCI performance which is assessed by classification accuracy and information transfer rate (ITR). Furthermore, we analyzed the waveforms of three patterns. Methods We compared three patterns in which semitransparent face is overlaid three primary colors as stimuli: red semitransparent face (RSF), green semitransparent face (GSF), and blue semitransparent face (BSF). Bayesian linear discriminant analysis (BLDA) was used to construct the individual classifier model. In addition, a Repeated-measures ANOVA (RM-ANOVA) and Bonferroni correction were chosen for statistical analysis. Results The results indicated that the RSF pattern achieved the highest online averaged accuracy with 93.89%, followed by the GSF pattern with 87.78%, while the lowest performance was caused by the BSF pattern with an accuracy of 81.39%. Furthermore, significant differences in classification accuracy and ITR were found between RSF and GSF (p < 0.05) and between RSF and BSF patterns (p < 0.05). Conclusion The semitransparent faces colored red (RSF) pattern yielded the best performance of the three patterns. The proposed patterns based on ERP-BCI system have a clinically significant impact by increasing communication speed and accuracy of the P300-speller for patients with severe motor impairment.
Collapse
Affiliation(s)
- Shurui Li
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Jing Jin
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Ian Daly
- Brain-Computer Interfacing and Neural Engineering Laboratory, School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom
| | - Cili Zuo
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Xingyu Wang
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Andrzej Cichocki
- Skolkowo Institute of Science and Technology, Moscow, Russia.,Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland.,Department of Informatics, Nicolaus Copernicus University, Toruń, Poland
| |
Collapse
|
14
|
Sosulski J, Tangermann M. Extremely Reduced Data Sets Indicate Optimal Stimulation Parameters for Classification in Brain-Computer Interfaces. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2019:2256-2260. [PMID: 31946349 DOI: 10.1109/embc.2019.8857460] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
The time between the onset of subsequent auditory or visual stimuli - also known as stimulus onset asynchrony (SOA) - determines many of the event-related potential characteristics of the resulting evoked brain signals. Specifically, the SOA value influences the performance of an individual subject in brain-computer interface (BCI) applications like spellers. In the past, subject-specific optimization of the SOA was rarely considered in BCI studies. Our research strives to reduce the time requirements of individual BCI stimulus parameter optimization. This work contributes to this goal in two ways. First, we show that even the classification performance on extremely reduced training data subsets reveals the influence of SOA. Second, we show, that these noisy estimates are sufficient to make decisions for individual choices of the SOA that transfer to good classification performance on large training data sets. Thus our work contributes to individually tailored SOA selection procedures for BCI users.
Collapse
|
15
|
Jin J, Li S, Daly I, Miao Y, Liu C, Wang X, Cichocki A. The Study of Generic Model Set for Reducing Calibration Time in P300-Based Brain–Computer Interface. IEEE Trans Neural Syst Rehabil Eng 2020; 28:3-12. [DOI: 10.1109/tnsre.2019.2956488] [Citation(s) in RCA: 69] [Impact Index Per Article: 17.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
16
|
Kshirsagar GB, Londhe ND. Improving Performance of Devanagari Script Input-Based P300 Speller Using Deep Learning. IEEE Trans Biomed Eng 2019; 66:2992-3005. [DOI: 10.1109/tbme.2018.2875024] [Citation(s) in RCA: 21] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
17
|
Cheng J, Jin J, Daly I, Zhang Y, Wang B, Wang X, Cichocki A. Effect of a combination of flip and zooming stimuli on the performance of a visual brain-computer interface for spelling. ACTA ACUST UNITED AC 2019; 64:29-38. [PMID: 29432199 DOI: 10.1515/bmt-2017-0082] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2017] [Accepted: 10/04/2017] [Indexed: 11/15/2022]
Abstract
Brain-computer interface (BCI) systems can allow their users to communicate with the external world by recognizing intention directly from their brain activity without the assistance of the peripheral motor nervous system. The P300-speller is one of the most widely used visual BCI applications. In previous studies, a flip stimulus (rotating the background area of the character) that was based on apparent motion, suffered from less refractory effects. However, its performance was not improved significantly. In addition, a presentation paradigm that used a "zooming" action (changing the size of the symbol) has been shown to evoke relatively higher P300 amplitudes and obtain a better BCI performance. To extend this method of stimuli presentation within a BCI and, consequently, to improve BCI performance, we present a new paradigm combining both the flip stimulus with a zooming action. This new presentation modality allowed BCI users to focus their attention more easily. We investigated whether such an action could combine the advantages of both types of stimuli presentation to bring a significant improvement in performance compared to the conventional flip stimulus. The experimental results showed that the proposed paradigm could obtain significantly higher classification accuracies and bit rates than the conventional flip paradigm (p<0.01).
Collapse
Affiliation(s)
- Jiao Cheng
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Jing Jin
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Ian Daly
- Brain-Computer Interfaces and Neural Engineering Laboratory, School of Computer Science and Electronic Engineering, University of Essex, Wivenhoe Park, Colchester, Essex, CO4 3SQ, UK
| | - Yu Zhang
- Department of Psychiatry and Behavior Sciences, Stanford University, Stanford, CA, USA
| | - Bei Wang
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Xingyu Wang
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Andrzej Cichocki
- Laboratory for Advanced Brain Signal Processing, Brain Science Institute, RIKEN, Wako-shi, Japan.,Skolkovo Institute of Science and Technology, Moscow, Russia.,Nicolaus Copernicus University (UMK), Torun, Poland
| |
Collapse
|
18
|
Guo M, Jin J, Jiao Y, Wang X, Cichockia A. Investigation of Visual Stimulus With Various Colors and the Layout for the Oddball Paradigm in Evoked Related Potential-Based Brain-Computer Interface. Front Comput Neurosci 2019; 13:24. [PMID: 31105544 PMCID: PMC6499038 DOI: 10.3389/fncom.2019.00024] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2019] [Accepted: 04/01/2019] [Indexed: 11/23/2022] Open
Abstract
Objective: Stimulus visual patterns, such as size, content, color, luminosity, and interval, play key roles for brain–computer interface (BCI) performance. However, the three primary colors to be intercompared as a single variable or factor on the same platform are poorly studied. In this work, we configured the visual stimulus patterns with red, green, and blue operating on a newly designed layout of the flash pattern of BCI to study the waveforms and performance of the evoked related potential (ERP). Approach: Twelve subjects participated in our experiment, and each subject was required to finish three different color sub-experiments. Four blocks of the interface were presented along the edge of the screen, and the other four were assembled in the center, aiming to investigate the problem of adjacency distraction. Repeated-measures ANOVA and Bonferroni correction were applied for statistical analysis. Main results: The averaged online accuracy was 98.44% for the red paradigm, higher than 92.71% for the green paradigm, and 93.23% for the blue paradigm. Furthermore, significant differences in online accuracy (p < 0.05) and information transfer rate (p < 0.05) were found between the red and green paradigms. Significance: The red stimulus paradigm yielded the best performance. The proposed design of ERP-based BCI was practical and effective for many potential applications.
Collapse
Affiliation(s)
- Miaoji Guo
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Jing Jin
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Yong Jiao
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Xingyu Wang
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Andrzej Cichockia
- Skolkowo Institute of Science and Technology (SKOLTECH), Moscow, Russia.,Systems Research Institute PAS, Warsaw, Poland.,Department of Informatics, Nicolaus Copernicus University (UMK), Torun, Poland
| |
Collapse
|
19
|
Mainsah BO, Reeves G, Collins LM, Throckmorton CS. Optimizing the stimulus presentation paradigm design for the P300-based brain-computer interface using performance prediction. J Neural Eng 2018; 14:046025. [PMID: 28548052 DOI: 10.1088/1741-2552/aa7525] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
OBJECTIVE The role of a brain-computer interface (BCI) is to discern a user's intended message or action by extracting and decoding relevant information from brain signals. Stimulus-driven BCIs, such as the P300 speller, rely on detecting event-related potentials (ERPs) in response to a user attending to relevant or target stimulus events. However, this process is error-prone because the ERPs are embedded in noisy electroencephalography (EEG) data, representing a fundamental problem in communication of the uncertainty in the information that is received during noisy transmission. A BCI can be modeled as a noisy communication system and an information-theoretic approach can be exploited to design a stimulus presentation paradigm to maximize the information content that is presented to the user. However, previous methods that focused on designing error-correcting codes failed to provide significant performance improvements due to underestimating the effects of psycho-physiological factors on the P300 ERP elicitation process and a limited ability to predict online performance with their proposed methods. Maximizing the information rate favors the selection of stimulus presentation patterns with increased target presentation frequency, which exacerbates refractory effects and negatively impacts performance within the context of an oddball paradigm. An information-theoretic approach that seeks to understand the fundamental trade-off between information rate and reliability is desirable. APPROACH We developed a performance-based paradigm (PBP) by tuning specific parameters of the stimulus presentation paradigm to maximize performance while minimizing refractory effects. We used a probabilistic-based performance prediction method as an evaluation criterion to select a final configuration of the PBP. MAIN RESULTS With our PBP, we demonstrate statistically significant improvements in online performance, both in accuracy and spelling rate, compared to the conventional row-column paradigm. SIGNIFICANCE By accounting for refractory effects, an information-theoretic approach can be exploited to significantly improve BCI performance across a wide range of performance levels.
Collapse
Affiliation(s)
- B O Mainsah
- Department of Electrical and Computer Engineering, Duke University, Durham, NC, United States of America
| | | | | | | |
Collapse
|
20
|
Mowla MR, Huggins JE, Natarajan B, Thompson DE. P300 Latency Estimation Using Least Mean Squares Filter. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2018; 2018:1976-1979. [PMID: 30440786 DOI: 10.1109/embc.2018.8512644] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Event-related potentials (ERPs) are the brain response directly related to specific events or stimuli. The P300 ERP is a positive deflection nominally 300ms post-stimulus that is related to mental decision making processes and also used in P300-based speller systems. Single-trial estimation of P300 responses will help to understand the underlying cognitive process more precisely and also to improve the speed of speller brain-computer interfaces (BCIs). This paper aims to develop a single-trial estimation of the P300 amplitudes and latencies by using the least mean squares (LMS) adaptive filtering method. Results for real data from people with amyotrophic lateral sclerosis (ALS) have shown that the LMS filter can be effectively used to estimate P300 latencies.
Collapse
|
21
|
Sugi M, Hagimoto Y, Nambu I, Gonzalez A, Takei Y, Yano S, Hokari H, Wada Y. Improving the Performance of an Auditory Brain-Computer Interface Using Virtual Sound Sources by Shortening Stimulus Onset Asynchrony. Front Neurosci 2018; 12:108. [PMID: 29535602 PMCID: PMC5835086 DOI: 10.3389/fnins.2018.00108] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2017] [Accepted: 02/12/2018] [Indexed: 12/03/2022] Open
Abstract
Recently, a brain-computer interface (BCI) using virtual sound sources has been proposed for estimating user intention via electroencephalogram (EEG) in an oddball task. However, its performance is still insufficient for practical use. In this study, we examine the impact that shortening the stimulus onset asynchrony (SOA) has on this auditory BCI. While very short SOA might improve its performance, sound perception and task performance become difficult, and event-related potentials (ERPs) may not be induced if the SOA is too short. Therefore, we carried out behavioral and EEG experiments to determine the optimal SOA. In the experiments, participants were instructed to direct attention to one of six virtual sounds (target direction). We used eight different SOA conditions: 200, 300, 400, 500, 600, 700, 800, and 1,100 ms. In the behavioral experiment, we recorded participant behavioral responses to target direction and evaluated recognition performance of the stimuli. In all SOA conditions, recognition accuracy was over 85%, indicating that participants could recognize the target stimuli correctly. Next, using a silent counting task in the EEG experiment, we found significant differences between target and non-target sound directions in all but the 200-ms SOA condition. When we calculated an identification accuracy using Fisher discriminant analysis (FDA), the SOA could be shortened by 400 ms without decreasing the identification accuracies. Thus, improvements in performance (evaluated by BCI utility) could be achieved. On average, higher BCI utilities were obtained in the 400 and 500-ms SOA conditions. Thus, auditory BCI performance can be optimized for both behavioral and neurophysiological responses by shortening the SOA.
Collapse
Affiliation(s)
- Miho Sugi
- Graduate School of Engineering, Nagaoka University of Technology, Nagaoka, Japan
| | - Yutaka Hagimoto
- Graduate School of Engineering, Nagaoka University of Technology, Nagaoka, Japan
| | - Isao Nambu
- Graduate School of Engineering, Nagaoka University of Technology, Nagaoka, Japan
| | - Alejandro Gonzalez
- Graduate School of Engineering, Nagaoka University of Technology, Nagaoka, Japan
| | - Yoshinori Takei
- Department of Electrical and Information Engineering, National Institute of Technology, Akita College, Akita, Japan
| | - Shohei Yano
- Department of Electrical and Electronic Systems Engineering, National Institute of Technology, Nagaoka College, Nagaoka, Japan
| | - Haruhide Hokari
- Graduate School of Engineering, Nagaoka University of Technology, Nagaoka, Japan
| | - Yasuhiro Wada
- Graduate School of Engineering, Nagaoka University of Technology, Nagaoka, Japan
| |
Collapse
|
22
|
Huang M, Jin J, Zhang Y, Hu D, Wang X. Usage of drip drops as stimuli in an auditory P300 BCI paradigm. Cogn Neurodyn 2017; 12:85-94. [PMID: 29435089 DOI: 10.1007/s11571-017-9456-y] [Citation(s) in RCA: 24] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2017] [Revised: 07/17/2017] [Accepted: 10/10/2017] [Indexed: 11/28/2022] Open
Abstract
Recently, many auditory BCIs are using beeps as auditory stimuli, while beeps sound unnatural and unpleasant for some people. It is proved that natural sounds make people feel comfortable, decrease fatigue, and improve the performance of auditory BCI systems. Drip drop is a kind of natural sounds that makes humans feel relaxed and comfortable. In this work, three kinds of drip drops were used as stimuli in an auditory-based BCI system to improve the user-friendness of the system. This study explored whether drip drops could be used as stimuli in the auditory BCI system. The auditory BCI paradigm with drip-drop stimuli, which was called the drip-drop paradigm (DP), was compared with the auditory paradigm with beep stimuli, also known as the beep paradigm (BP), in items of event-related potential amplitudes, online accuracies and scores on the likability and difficulty to demonstrate the advantages of DP. DP obtained significantly higher online accuracy and information transfer rate than the BP (p < 0.05, Wilcoxon signed test; p < 0.05, Wilcoxon signed test). Besides, DP obtained higher scores on the likability with no significant difference on the difficulty (p < 0.05, Wilcoxon signed test). The results showed that the drip drops were reliable acoustic materials as stimuli in an auditory BCI system.
Collapse
Affiliation(s)
- Minqiang Huang
- 1Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, People's Republic of China
| | - Jing Jin
- 1Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, People's Republic of China
| | - Yu Zhang
- 1Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, People's Republic of China
| | - Dewen Hu
- 2College of Mechatronics and Automation, National University of Defense Technology, Changsha, Hunan 410073 People's Republic of China
| | - Xingyu Wang
- 1Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, People's Republic of China
| |
Collapse
|
23
|
Carabalona R. The Role of the Interplay between Stimulus Type and Timing in Explaining BCI-Illiteracy for Visual P300-Based Brain-Computer Interfaces. Front Neurosci 2017; 11:363. [PMID: 28713233 PMCID: PMC5492449 DOI: 10.3389/fnins.2017.00363] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2016] [Accepted: 06/12/2017] [Indexed: 11/13/2022] Open
Abstract
Visual P300-based Brain-Computer Interface (BCI) spellers enable communication or interaction with the environment by flashing elements in a matrix and exploiting consequent changes in end-user's brain activity. Despite research efforts, performance variability and BCI-illiteracy still are critical issues for real world applications. Moreover, there is a quite unaddressed kind of BCI-illiteracy, which becomes apparent when the same end-user operates BCI-spellers intended for different applications: our aim is to understand why some well performers can become BCI-illiterate depending on speller type. We manipulated stimulus type (factor STIM: either characters or icons), color (factor COLOR: white, green) and timing (factor SPEED: fast, slow). Each BCI session consisted of training (without feedback) and performance phase (with feedback), both in copy-spelling. For fast flashing spellers, we observed a performance worsening for white icon-speller. Our findings are consistent with existing results reported on end-users using identical white×fast spellers, indicating independence of worsening trend from users' group. The use of slow stimulation timing shed a new light on the perceptual and cognitive phenomena related to the use of a BCI-speller during both the training and the performance phase. We found a significant STIM main effect for the N1 component on P z and PO7 during the training phase and on PO8 during the performance phase, whereas in both phases neither the STIM×COLOR interaction nor the COLOR main effect was statistically significant. After collapsing data for factor COLOR, it emerged a statistically significant modulation of N1 amplitude depending to the phase of BCI session: N1 was more negative for icons than for characters both on P z and PO7 (training), whereas the opposite modulation was observed for PO8 (performance). Results indicate that both feedback and expertise with respect to the stimulus type can modulate the N1 component and that icons require more perceptual analysis. Therefore, fast flashing is likely to be more detrimental for end-users' performance in case of icon-spellers. In conclusion, the interplay between stimulus type and timing seems relevant for a satisfactory and efficient end-user's BCI-experience.
Collapse
Affiliation(s)
- Roberta Carabalona
- Biomedical Technological Department, Fondazione Don Carlo Gnocchi Onlus (IRCCS)Milan, Italy
| |
Collapse
|
24
|
Ma Z, Qiu T. Performance improvement of ERP-based brain–computer interface via varied geometric patterns. Med Biol Eng Comput 2017; 55:2245-2256. [DOI: 10.1007/s11517-017-1671-5] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2016] [Accepted: 06/19/2017] [Indexed: 11/24/2022]
|
25
|
Jin J, Zhang H, Daly I, Wang X, Cichocki A. An improved P300 pattern in BCI to catch user's attention. J Neural Eng 2017; 14:036001. [PMID: 28224970 DOI: 10.1088/1741-2552/aa6213] [Citation(s) in RCA: 40] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
OBJECTIVE Brain-computer interfaces (BCIs) can help patients who have lost control over most muscles but are still conscious and able to communicate or interact with the environment. One of the most popular types of BCI is the P300-based BCI. With this BCI, users are asked to count the number of appearances of target stimuli in an experiment. To date, the majority of visual P300-based BCI systems developed have used the same character or picture as the target for every stimulus presentation, which can bore users. Consequently, users attention may decrease or be negatively affected by adjacent stimuli. APPROACH In this study, a new stimulus is presented to increase user concentration. Honeycomb-shaped figures with 1-3 red dots were used as stimuli. The number and the positions of the red dots in the honeycomb-shaped figure were randomly changed during BCI control. The user was asked to count the number of the dots presented in each flash instead of the number of times they flashed. To assess the performance of this new stimulus, another honeycomb-shaped stimulus, without red dots, was used as a control condition. MAIN RESULTS The results showed that the honeycomb-shaped stimuli with red dots obtained significantly higher classification accuracies and information transfer rates (p < 0.05) compared to the honeycomb-shaped stimulus without red dots. SIGNIFICANCE The results indicate that this proposed method can be a promising approach to improve the performance of the BCI system and can be an efficient method in daily application.
Collapse
Affiliation(s)
- Jing Jin
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, People's Republic of China
| | | | | | | | | |
Collapse
|
26
|
Zhou S, Jin J, Daly I, Wang X, Cichocki A. Optimizing the Face Paradigm of BCI System by Modified Mismatch Negative Paradigm. Front Neurosci 2016; 10:444. [PMID: 27774046 PMCID: PMC5054457 DOI: 10.3389/fnins.2016.00444] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2016] [Accepted: 09/14/2016] [Indexed: 11/13/2022] Open
Abstract
Many recent studies have focused on improving the performance of event-related potential (ERP) based brain computer interfaces (BCIs). The use of a face pattern has been shown to obtain high classification accuracies and information transfer rates (ITRs) by evoking discriminative ERPs (N200 and N400) in addition to P300 potentials. Recently, it has been proved that the performance of traditional P300-based BCIs could be improved through a modification of the mismatch pattern. In this paper, a mismatch inverted face pattern (MIF-pattern) was presented to improve the performance of the inverted face pattern (IF-pattern), one of the state of the art patterns used in visual-based BCI systems. Ten subjects attended in this experiment. The result showed that the mismatch inverted face pattern could evoke significantly larger vertex positive potentials (p < 0.05) and N400s (p < 0.05) compared to the inverted face pattern. The classification accuracy (mean accuracy is 99.58%) and ITRs (mean bit rate is 27.88 bit/min) of the mismatch inverted face pattern was significantly higher than that of the inverted face pattern (p < 0.05).
Collapse
Affiliation(s)
- Sijie Zhou
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology Shanghai, China
| | - Jing Jin
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology Shanghai, China
| | - Ian Daly
- Brain Embodiment Lab, School of Systems Engineering, University of Reading Reading, UK
| | - Xingyu Wang
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology Shanghai, China
| | - Andrzej Cichocki
- Laboratory for Advanced Brain Signal Processing, Brain Science Institute, RIKENWako-shi, Japan; Systems Research Institute PAS, Warsaw and Nicolaus Copernicus University (UMK)Torun, Poland
| |
Collapse
|
27
|
Gibson RM, Owen AM, Cruse D. Brain-computer interfaces for patients with disorders of consciousness. PROGRESS IN BRAIN RESEARCH 2016; 228:241-91. [PMID: 27590972 DOI: 10.1016/bs.pbr.2016.04.003] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Abstract
The disorders of consciousness refer to clinical conditions that follow a severe head injury. Patients diagnosed as in a vegetative state lack awareness, while patients diagnosed as in a minimally conscious state retain fluctuating awareness. However, it is a challenge to accurately diagnose these disorders with clinical assessments of behavior. To improve diagnostic accuracy, neuroimaging-based approaches have been developed to detect the presence or absence of awareness in patients who lack overt responsiveness. For the small subset of patients who retain awareness, brain-computer interfaces could serve as tools for communication and environmental control. Here we review the existing literature concerning the sensory and cognitive abilities of patients with disorders of consciousness with respect to existing brain-computer interface designs. We highlight the challenges of device development for this special population and address some of the most promising approaches for future investigations.
Collapse
Affiliation(s)
- R M Gibson
- The Brain and Mind Institute, University of Western Ontario, London, ON, Canada; University of Western Ontario, London, ON, Canada.
| | - A M Owen
- The Brain and Mind Institute, University of Western Ontario, London, ON, Canada; University of Western Ontario, London, ON, Canada
| | - D Cruse
- School of Psychology, University of Birmingham, Birmingham, United Kingdom
| |
Collapse
|
28
|
da Silva-Sauer L, Valero-Aguayo L, de la Torre-Luque A, Ron-Angevin R, Varona-Moya S. Concentration on performance with P300-based BCI systems: a matter of interface features. APPLIED ERGONOMICS 2016; 52:325-332. [PMID: 26360225 DOI: 10.1016/j.apergo.2015.08.002] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/29/2014] [Revised: 08/02/2015] [Accepted: 08/10/2015] [Indexed: 06/05/2023]
Abstract
People who suffer from severe motor disabilities have difficulties to communicate with others or to interact with their environment using natural, i.e., muscular channels. These limitations can be overcome to some extent by using brain-computer interfaces (BCIs), because such systems allow users to communicate on the basis of their brain activity only. Among the several types of BCIs for spelling purposes, those that rely on the P300 event related potential-P300-based spellers-are chosen preferentially due to their high reliability. However, they demand from the user to sustain his/her attention to the desired character over a relatively long period of time. Therefore, the user's capacity to concentrate can affect his/her performance with a P300-based speller. The aim of this study was to test this hypothesis using three different interfaces: one based on the classic P300 speller paradigm, another also based on that speller but including a word predictor, and a third one that was based on the T9 interface developed for mobile phones. User performance was assessed by measuring the time to complete a spelling task and the accuracy of character selection. The d2 test was applied to assess attention and concentration. Sample (N = 14) was divided into two groups basing on of concentration scores. As a result, performance was better with the predictor-enriched interfaces: less time was needed to solve the task and participants made fewer errors (p < .05). There were also significant effects of concentration (p < .05) on performance with the standard P300 speller. In conclusion, the performance of those users with lower concentration level can be improved by providing BCIs with more interactive interfaces. These findings provide substantial evidence in order to highlight the impact of psychological features on BCI performance and should be taken into account for future assistive technology systems.
Collapse
Affiliation(s)
- Leandro da Silva-Sauer
- Depto. de Personalidad, Evaluación y Tratamiento Psicológico, Facultad de Psicología, Universidad de Málaga, Spain; Depto. de Tecnología Electrónica, Escuela Técnica Superior de Ingeniería de Telecomunicaciones, Universidad de Málaga, Spain.
| | - Luis Valero-Aguayo
- Depto. de Personalidad, Evaluación y Tratamiento Psicológico, Facultad de Psicología, Universidad de Málaga, Spain
| | - Alejandro de la Torre-Luque
- Institut Universitari d'Investigació en Ciències de la Salut (IUNICS), University of the Balearic Islands, Spain
| | - Ricardo Ron-Angevin
- Depto. de Tecnología Electrónica, Escuela Técnica Superior de Ingeniería de Telecomunicaciones, Universidad de Málaga, Spain
| | - Sergio Varona-Moya
- Depto. de Tecnología Electrónica, Escuela Técnica Superior de Ingeniería de Telecomunicaciones, Universidad de Málaga, Spain
| |
Collapse
|
29
|
Cecotti H. Toward shift invariant detection of event-related potentials in non-invasive brain-computer interface. Pattern Recognit Lett 2015. [DOI: 10.1016/j.patrec.2015.01.015] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
30
|
Jin J, Sellers EW, Zhou S, Zhang Y, Wang X, Cichocki A. A P300 brain-computer interface based on a modification of the mismatch negativity paradigm. Int J Neural Syst 2015; 25:1550011. [PMID: 25804352 DOI: 10.1142/s0129065715500112] [Citation(s) in RCA: 106] [Impact Index Per Article: 11.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
The P300-based brain-computer interface (BCI) is an extension of the oddball paradigm, and can facilitate communication for people with severe neuromuscular disorders. It has been shown that, in addition to the P300, other event-related potential (ERP) components have been shown to contribute to successful operation of the P300 BCI. Incorporating these components into the classification algorithm can improve the classification accuracy and information transfer rate (ITR). In this paper, a single character presentation paradigm was compared to a presentation paradigm that is based on the visual mismatch negativity. The mismatch negativity paradigm showed significantly higher classification accuracy and ITRs than a single character presentation paradigm. In addition, the mismatch paradigm elicited larger N200 and N400 components than the single character paradigm. The components elicited by the presentation method were consistent with what would be expected from a mismatch paradigm and a typical P300 was also observed. The results show that increasing the signal-to-noise ratio by increasing the amplitude of ERP components can significantly improve BCI speed and accuracy. The mismatch presentation paradigm may be considered a viable option to the traditional P300 BCI paradigm.
Collapse
Affiliation(s)
- Jing Jin
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai 200237, China
| | | | | | | | | | | |
Collapse
|
31
|
Yeom SK, Fazli S, Müller KR, Lee SW. An efficient ERP-based brain-computer interface using random set presentation and face familiarity. PLoS One 2014; 9:e111157. [PMID: 25384045 PMCID: PMC4226481 DOI: 10.1371/journal.pone.0111157] [Citation(s) in RCA: 69] [Impact Index Per Article: 6.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2014] [Accepted: 09/27/2014] [Indexed: 11/18/2022] Open
Abstract
Event-related potential (ERP)-based P300 spellers are commonly used in the field of brain-computer interfaces as an alternative channel of communication for people with severe neuro-muscular diseases. This study introduces a novel P300 based brain-computer interface (BCI) stimulus paradigm using a random set presentation pattern and exploiting the effects of face familiarity. The effect of face familiarity is widely studied in the cognitive neurosciences and has recently been addressed for the purpose of BCI. In this study we compare P300-based BCI performances of a conventional row-column (RC)-based paradigm with our approach that combines a random set presentation paradigm with (non-) self-face stimuli. Our experimental results indicate stronger deflections of the ERPs in response to face stimuli, which are further enhanced when using the self-face images, and thereby improving P300-based spelling performance. This lead to a significant reduction of stimulus sequences required for correct character classification. These findings demonstrate a promising new approach for improving the speed and thus fluency of BCI-enhanced communication with the widely used P300-based BCI setup.
Collapse
Affiliation(s)
- Seul-Ki Yeom
- Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea
| | - Siamac Fazli
- Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea
| | - Klaus-Robert Müller
- Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea
- Machine Learning Group, Berlin Institute of Technology, Berlin, Germany
| | - Seong-Whan Lee
- Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea
- * E-mail:
| |
Collapse
|
32
|
Alcaide-Aguirre RE, Huggins JE. Novel hold-release functionality in a P300 brain-computer interface. J Neural Eng 2014; 11:066010. [PMID: 25380071 DOI: 10.1088/1741-2560/11/6/066010] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
Assistive technology control interface theory describes interface activation and interface deactivation as distinct properties of any control interface. Separating control of activation and deactivation allows precise timing of the duration of the activation. Objective. We propose a novel P300 brain-computer interface (BCI) functionality with separate control of the initial activation and the deactivation (hold-release) of a selection. Approach. Using two different layouts and off-line analysis, we tested the accuracy with which subjects could (1) hold their selection and (2) quickly change between selections. Main results. Mean accuracy across all subjects for the hold-release algorithm was 85% with one hold-release classification and 100% with two hold-release classifications. Using a layout designed to lower perceptual errors, accuracy increased to a mean of 90% and the time subjects could hold a selection was 40% longer than with the standard layout. Hold-release functionality provides improved response time (6-16 times faster) over the initial P300 BCI selection by allowing the BCI to make hold-release decisions from very few flashes instead of after multiple sequences of flashes. Significance. For the BCI user, hold-release functionality allows for faster, more continuous control with a P300 BCI, creating new options for BCI applications.
Collapse
Affiliation(s)
- R E Alcaide-Aguirre
- University of Michigan, 500 S State St., Ann Arbor, MI 48109-2215, USA. Neuroscience Graduate Program, 4137 Undergraduate Science Building, 204 Washtenaw Avenue, Ann Arbor, MI 48109-2215, USA
| | | |
Collapse
|
33
|
Ikegami S, Takano K, Kondo K, Saeki N, Kansaku K. A region-based two-step P300-based brain–computer interface for patients with amyotrophic lateral sclerosis. Clin Neurophysiol 2014; 125:2305-2312. [PMID: 24731767 DOI: 10.1016/j.clinph.2014.03.013] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2013] [Revised: 03/09/2014] [Accepted: 03/11/2014] [Indexed: 11/16/2022]
Affiliation(s)
- Shiro Ikegami
- Systems Neuroscience Section, Department of Rehabilitation for Brain Functions, Research Institute of National Rehabilitation Center for Persons with Disabilities, Tokorozawa, Saitama 359-8555, Japan; Department of Neurological Surgery, Chiba University Graduate School of Medicine, Chiba, Chiba 260-8670, Japan
| | - Kouji Takano
- Systems Neuroscience Section, Department of Rehabilitation for Brain Functions, Research Institute of National Rehabilitation Center for Persons with Disabilities, Tokorozawa, Saitama 359-8555, Japan
| | - Kiyohiko Kondo
- Department of Neurology, Yoka Hospital, Yabu, Hyogo 667-8555, Japan
| | - Naokatsu Saeki
- Department of Neurological Surgery, Chiba University Graduate School of Medicine, Chiba, Chiba 260-8670, Japan
| | - Kenji Kansaku
- Systems Neuroscience Section, Department of Rehabilitation for Brain Functions, Research Institute of National Rehabilitation Center for Persons with Disabilities, Tokorozawa, Saitama 359-8555, Japan; Center for Frontier Medical Engineering, Chiba University, Chiba 263-0022, Japan; Brain Science Inspired Life Support Research Center, The University of Electro-Communications, Tokyo 182-8585, Japan.
| |
Collapse
|
34
|
Zhou Z, Yin E, Liu Y, Jiang J, Hu D. A novel task-oriented optimal design for P300-based brain-computer interfaces. J Neural Eng 2014; 11:056003. [PMID: 25080373 DOI: 10.1088/1741-2560/11/5/056003] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
Objective. The number of items of a P300-based brain-computer interface (BCI) should be adjustable in accordance with the requirements of the specific tasks. To address this issue, we propose a novel task-oriented optimal approach aimed at increasing the performance of general P300 BCIs with different numbers of items. Approach. First, we proposed a stimulus presentation with variable dimensions (VD) paradigm as a generalization of the conventional single-character (SC) and row-column (RC) stimulus paradigms. Furthermore, an embedding design approach was employed for any given number of items. Finally, based on the score-P model of each subject, the VD flash pattern was selected by a linear interpolation approach for a certain task. Main results. The results indicate that the optimal BCI design consistently outperforms the conventional approaches, i.e., the SC and RC paradigms. Specifically, there is significant improvement in the practical information transfer rate for a large number of items. Significance. The results suggest that the proposed optimal approach would provide useful guidance in the practical design of general P300-based BCIs.
Collapse
Affiliation(s)
- Zongtan Zhou
- Department of Automatic Control, College of Mechatronic Engineering and Automation, National University of Defense Technology, 410073 Changsha, Hunan, People's Republic of China
| | | | | | | | | |
Collapse
|
35
|
Kindermans PJ, Schreuder M, Schrauwen B, Müller KR, Tangermann M. True zero-training brain-computer interfacing--an online study. PLoS One 2014; 9:e102504. [PMID: 25068464 PMCID: PMC4113217 DOI: 10.1371/journal.pone.0102504] [Citation(s) in RCA: 54] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2014] [Accepted: 06/19/2014] [Indexed: 11/18/2022] Open
Abstract
Despite several approaches to realize subject-to-subject transfer of pre-trained classifiers, the full performance of a Brain-Computer Interface (BCI) for a novel user can only be reached by presenting the BCI system with data from the novel user. In typical state-of-the-art BCI systems with a supervised classifier, the labeled data is collected during a calibration recording, in which the user is asked to perform a specific task. Based on the known labels of this recording, the BCI's classifier can learn to decode the individual's brain signals. Unfortunately, this calibration recording consumes valuable time. Furthermore, it is unproductive with respect to the final BCI application, e.g. text entry. Therefore, the calibration period must be reduced to a minimum, which is especially important for patients with a limited concentration ability. The main contribution of this manuscript is an online study on unsupervised learning in an auditory event-related potential (ERP) paradigm. Our results demonstrate that the calibration recording can be bypassed by utilizing an unsupervised trained classifier, that is initialized randomly and updated during usage. Initially, the unsupervised classifier tends to make decoding mistakes, as the classifier might not have seen enough data to build a reliable model. Using a constant re-analysis of the previously spelled symbols, these initially misspelled symbols can be rectified posthoc when the classifier has learned to decode the signals. We compare the spelling performance of our unsupervised approach and of the unsupervised posthoc approach to the standard supervised calibration-based dogma for n = 10 healthy users. To assess the learning behavior of our approach, it is unsupervised trained from scratch three times per user. Even with the relatively low SNR of an auditory ERP paradigm, the results show that after a limited number of trials (30 trials), the unsupervised approach performs comparably to a classic supervised model.
Collapse
Affiliation(s)
- Pieter-Jan Kindermans
- Electronics and Information Systems (ELIS) Dept., Ghent University, Ghent, Belgium
- * E-mail: (PJK); (KRM); (MT)
| | - Martijn Schreuder
- Machine Learning Laboratory, Technical University of Berlin, Berlin, Germany
| | - Benjamin Schrauwen
- Electronics and Information Systems (ELIS) Dept., Ghent University, Ghent, Belgium
| | - Klaus-Robert Müller
- Machine Learning Laboratory, Technical University of Berlin, Berlin, Germany
- Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea
- * E-mail: (PJK); (KRM); (MT)
| | - Michael Tangermann
- BrainLinks-BrainTools Excellence Cluster, Computer Science Dept., University of Freiburg, Freiburg, Germany
- * E-mail: (PJK); (KRM); (MT)
| |
Collapse
|
36
|
Ganin IP, Shishkin SL, Kaplan AY. A P300-based brain-computer interface with stimuli on moving objects: four-session single-trial and triple-trial tests with a game-like task design. PLoS One 2013; 8:e77755. [PMID: 24302977 PMCID: PMC3840230 DOI: 10.1371/journal.pone.0077755] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2013] [Accepted: 09/08/2013] [Indexed: 11/24/2022] Open
Abstract
Brain-computer interfaces (BCIs) are tools for controlling computers and other devices without using muscular activity, employing user-controlled variations in signals recorded from the user's brain. One of the most efficient noninvasive BCIs is based on the P300 wave of the brain's response to stimuli and is therefore referred to as the P300 BCI. Many modifications of this BCI have been proposed to further improve the BCI's characteristics or to better adapt the BCI to various applications. However, in the original P300 BCI and in all of its modifications, the spatial positions of stimuli were fixed relative to each other, which can impose constraints on designing applications controlled by this BCI. We designed and tested a P300 BCI with stimuli presented on objects that were freely moving on a screen at a speed of 5.4°/s. Healthy participants practiced a game-like task with this BCI in either single-trial or triple-trial mode within four sessions. At each step, the participants were required to select one of nine moving objects. The mean online accuracy of BCI-based selection was 81% in the triple-trial mode and 65% in the single-trial mode. A relatively high P300 amplitude was observed in response to targets in most participants. Self-rated interest in the task was high and stable over the four sessions (the medians in the 1st/4th sessions were 79/84% and 76/71% in the groups practicing in the single-trial and triple-trial modes, respectively). We conclude that the movement of stimulus positions relative to each other may not prevent the efficient use of the P300 BCI by people controlling their gaze, e.g., in robotic devices and in video games.
Collapse
Affiliation(s)
- Ilya P. Ganin
- Laboratory for Neurophysiology and Neuro-Computer Interfaces, Faculty of Biology, Lomonosov Moscow State University, Moscow, Russia
| | - Sergei L. Shishkin
- Laboratory for Neurophysiology and Neuro-Computer Interfaces, Faculty of Biology, Lomonosov Moscow State University, Moscow, Russia
- Laboratory for Neuroergonomics and Brain-Computer Interfaces, Centre of Converging of Nano-, Bio-, Information, Cognitive and Social Sciences and Technologies (NBICS Centre), National Research Centre “Kurchatov Institute”, Moscow, Russia
| | - Alexander Y. Kaplan
- Laboratory for Neurophysiology and Neuro-Computer Interfaces, Faculty of Biology, Lomonosov Moscow State University, Moscow, Russia
| |
Collapse
|
37
|
Yaming Xu, Nakajima Y. A Two-Level Predictive Event-Related Potential-Based Brain–Computer Interface. IEEE Trans Biomed Eng 2013; 60:2839-47. [DOI: 10.1109/tbme.2013.2265103] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
38
|
Kaplan AY, Shishkin SL, Ganin IP, Basyul IA, Zhigalov AY. Adapting the P300-Based Brain–Computer Interface for Gaming: A Review. IEEE TRANSACTIONS ON COMPUTATIONAL INTELLIGENCE AND AI IN GAMES 2013. [DOI: 10.1109/tciaig.2012.2237517] [Citation(s) in RCA: 58] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
39
|
Brain Control: Human-computer Integration Control Based on Brain-computer Interface Approach. ACTA ACUST UNITED AC 2013. [DOI: 10.1016/s1874-1029(13)60023-3] [Citation(s) in RCA: 28] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
40
|
Zhao Q, Zhang Y, Onishi A, Cichocki A. An Affective BCI Using Multiple ERP Components Associated to Facial Emotion Processing. SPRINGERBRIEFS IN ELECTRICAL AND COMPUTER ENGINEERING 2013. [DOI: 10.1007/978-3-642-36083-1_7] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/31/2023]
|
41
|
Jin J, Allison BZ, Kaufmann T, Kübler A, Zhang Y, Wang X, Cichocki A. The changing face of P300 BCIs: a comparison of stimulus changes in a P300 BCI involving faces, emotion, and movement. PLoS One 2012. [PMID: 23189154 PMCID: PMC3506655 DOI: 10.1371/journal.pone.0049688] [Citation(s) in RCA: 111] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022] Open
Abstract
BACKGROUND One of the most common types of brain-computer interfaces (BCIs) is called a P300 BCI, since it relies on the P300 and other event-related potentials (ERPs). In the canonical P300 BCI approach, items on a monitor flash briefly to elicit the necessary ERPs. Very recent work has shown that this approach may yield lower performance than alternate paradigms in which the items do not flash but instead change in other ways, such as moving, changing colour or changing to characters overlaid with faces. METHODOLOGY/PRINCIPAL FINDINGS The present study sought to extend this research direction by parametrically comparing different ways to change items in a P300 BCI. Healthy subjects used a P300 BCI across six different conditions. Three conditions were similar to our prior work, providing the first direct comparison of characters flashing, moving, and changing to faces. Three new conditions also explored facial motion and emotional expression. The six conditions were compared across objective measures such as classification accuracy and bit rate as well as subjective measures such as perceived difficulty. In line with recent studies, our results indicated that the character flash condition resulted in the lowest accuracy and bit rate. All four face conditions (mean accuracy >91%) yielded significantly better performance than the flash condition (mean accuracy = 75%). CONCLUSIONS/SIGNIFICANCE Objective results reaffirmed that the face paradigm is superior to the canonical flash approach that has dominated P300 BCIs for over 20 years. The subjective reports indicated that the conditions that yielded better performance were not considered especially burdensome. Therefore, although further work is needed to identify which face paradigm is best, it is clear that the canonical flash approach should be replaced with a face paradigm when aiming at increasing bit rate. However, the face paradigm has to be further explored with practical applications particularly with locked-in patients.
Collapse
Affiliation(s)
- Jing Jin
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, PR China.
| | | | | | | | | | | | | |
Collapse
|
42
|
Townsend G, Shanahan J, Ryan DB, Sellers EW. A general P300 brain-computer interface presentation paradigm based on performance guided constraints. Neurosci Lett 2012; 531:63-8. [PMID: 22960261 DOI: 10.1016/j.neulet.2012.08.041] [Citation(s) in RCA: 29] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2012] [Revised: 07/06/2012] [Accepted: 08/22/2012] [Indexed: 10/27/2022]
Abstract
An electroencephalographic-based brain-computer interface (BCI) can provide a non-muscular method of communication. A general model for P300-based BCI stimulus presentations is introduced--the "m choose n" or C(m (number of flashes per sequence), n (number of flashes per item)) paradigm, which is a universal extension of the previously reported checkerboard paradigm (CBP). C(m,n) captures all possible (unconstrained) ways to flash target items, and then applies constraints to enhance ERP's produced by attended matrix items. We explore a C(36,5) instance of C(m,n) called the "five flash paradigm" (FFP) and compare its performance to the CBP. Eight subjects were tested in each paradigm, counter-balanced. Twelve minutes of calibration data were used as input to a stepwise linear discriminant analysis to derive classification coefficients used for online classification. Accuracy was consistently high for FFP (88%) and CBP (90%); information transfer rate was significantly higher for the FFP (63 bpm) than the CBP (48 bpm). The C(m,n) is a novel and effective general strategy for organizing stimulus groups. Appropriate choices for "m," "n," and specific constraints can improve presentation paradigms by adjusting the parameters in a subject specific manner. This may be especially important for people with neuromuscular disabilities.
Collapse
Affiliation(s)
- George Townsend
- Algoma University, Sault Ste. Marie, Ontario P6A 2G4, Canada
| | | | | | | |
Collapse
|
43
|
Fazel-Rezai R, Allison BZ, Guger C, Sellers EW, Kleih SC, Kübler A. P300 brain computer interface: current challenges and emerging trends. FRONTIERS IN NEUROENGINEERING 2012; 5:14. [PMID: 22822397 PMCID: PMC3398470 DOI: 10.3389/fneng.2012.00014] [Citation(s) in RCA: 129] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/30/2012] [Accepted: 06/21/2012] [Indexed: 11/13/2022]
Abstract
A brain-computer interface (BCI) enables communication without movement based on brain signals measured with electroencephalography (EEG). BCIs usually rely on one of three types of signals: the P300 and other components of the event-related potential (ERP), steady state visual evoked potential (SSVEP), or event related desynchronization (ERD). Although P300 BCIs were introduced over twenty years ago, the past few years have seen a strong increase in P300 BCI research. This closed-loop BCI approach relies on the P300 and other components of the ERP, based on an oddball paradigm presented to the subject. In this paper, we overview the current status of P300 BCI technology, and then discuss new directions: paradigms for eliciting P300s; signal processing methods; applications; and hybrid BCIs. We conclude that P300 BCIs are quite promising, as several emerging directions have not yet been fully explored and could lead to improvements in bit rate, reliability, usability, and flexibility.
Collapse
Affiliation(s)
- Reza Fazel-Rezai
- Biomedical Signal and Image Processing Laboratory, Department of Electrical Engineering, University of North Dakota, Grand ForksND, USA
| | - Brendan Z. Allison
- Cognitive Neuroscience Laboratory, Department of Cognitive Science, University of California at San Diego, La JollaCA, USA
| | - Christoph Guger
- g.tec Medical Engineering GmbH/Guger Technologies OGGraz, Austria
| | - Eric W. Sellers
- ETSU Brain-Computer Interface Laboratory, East Tennessee State University, Johnson CityTN, USA
| | - Sonja C. Kleih
- Department of Psychology I, University of WürzburgWürzburg, Germany
| | - Andrea Kübler
- Department of Psychology I, University of WürzburgWürzburg, Germany
| |
Collapse
|
44
|
Chatelle C, Chennu S, Noirhomme Q, Cruse D, Owen AM, Laureys S. Brain-computer interfacing in disorders of consciousness. Brain Inj 2012; 26:1510-22. [PMID: 22759199 DOI: 10.3109/02699052.2012.698362] [Citation(s) in RCA: 60] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
Abstract
BACKGROUND Recent neuroimaging research has strikingly demonstrated the existence of covert awareness in some patients with disorders of consciousness (DoC). These findings have highlighted the potential for the development of simple brain-computer interfaces (BCI) as a diagnosis in behaviourally unresponsive patients. OBJECTIVES This study here reviews current EEG-based BCIs that hold potential for assessing and eventually assisting patients with DoC. It highlights key areas for further development that might eventually make their application feasible in this challenging patient group. METHODS The major types of BCIs proposed in the literature are considered, namely those based on the P3 potential, sensorimotor rhythms, steady state oscillations and slow cortical potentials. In each case, a brief overview of the relevant literature is provided and then their relative merits for BCI applications in DoC are considered. RESULTS A range of BCI designs have been proposed and tested for enabling communication in fully conscious, paralysed patients. Although many of these have potential applicability for patients with DoC, they share some key challenges that need to be overcome, including limitations of stimulation modality, feedback, user training and consistency. CONCLUSION Future work will need to address the technical and practical challenges facing reliable implementation at the patient's bedside.
Collapse
Affiliation(s)
- Camille Chatelle
- Coma Science Group, Cyclotron Research Centre, University of Liège, Liège, Belgium.
| | | | | | | | | | | |
Collapse
|
45
|
Cipresso P, Carelli L, Solca F, Meazzi D, Meriggi P, Poletti B, Lulé D, Ludolph AC, Silani V, Riva G. The use of P300-based BCIs in amyotrophic lateral sclerosis: from augmentative and alternative communication to cognitive assessment. Brain Behav 2012; 2:479-98. [PMID: 22950051 PMCID: PMC3432970 DOI: 10.1002/brb3.57] [Citation(s) in RCA: 37] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/13/2011] [Revised: 03/06/2012] [Accepted: 03/13/2012] [Indexed: 12/11/2022] Open
Abstract
The use of augmentative and alternative communication (AAC) tools in patients with amyotrophic lateral sclerosis (ALS), as effective means to compensate for the progressive loss of verbal and gestural communication, has been deeply investigated in the recent literature. The development of advanced AAC systems, such as eye-tracking (ET) and brain-computer interface (BCI) devices, allowed to bypass the important motor difficulties present in ALS patients. In particular, BCIs could be used in moderate to severe stages of the disease, since they do not require preserved ocular-motor ability, which is necessary for ET applications. Furthermore, some studies have proved the reliability of BCIs, regardless of the severity of the disease and the level of physical decline. However, the use of BCI in ALS patients still shows some limitations, related to both technical and neuropsychological issues. In particular, a range of cognitive deficits in most ALS patients have been observed. At the moment, no effective verbal-motor free measures are available for the evaluation of ALS patients' cognitive integrity; BCIs could offer a new possibility to administer cognitive tasks without the need of verbal or motor responses, as highlighted by preliminary studies in this field. In this review, we outline the essential features of BCIs systems, considering advantages and challenges of these tools with regard to ALS patients and the main applications developed in this field. We then outline the main findings with regard to cognitive deficits observed in ALS and some preliminary attempts to evaluate them by means of BCIs. The definition of specific cognitive profiles could help to draw flexible approaches tailored on patients' needs. It could improve BCIs efficacy and reduce patients' efforts. Finally, we handle the open question, represented by the use of BCIs with totally locked in patients, who seem unable to reliably learn to use such tool.
Collapse
Affiliation(s)
- Pietro Cipresso
- Applied Technology for Neuro‐Psychology Lab, IRCCS Istituto Auxologico Italiano, Milan, Italy
| | - Laura Carelli
- Department of Neurology and Laboratory of Neuroscience ‐ “Dino Ferrari” Center ‐ Università degli Studi di Milano ‐ IRCCS Istituto Auxologico Italiano, Milano, Italy
| | - Federica Solca
- Department of Neurology and Laboratory of Neuroscience ‐ “Dino Ferrari” Center ‐ Università degli Studi di Milano ‐ IRCCS Istituto Auxologico Italiano, Milano, Italy
| | - Daniela Meazzi
- Applied Technology for Neuro‐Psychology Lab, IRCCS Istituto Auxologico Italiano, Milan, Italy
| | - Paolo Meriggi
- Polo Tecnologico–Biomedical Technology Department, Fondazione Don Carlo Gnocchi Onlus, Milano, Italy
| | - Barbara Poletti
- Department of Neurology and Laboratory of Neuroscience ‐ “Dino Ferrari” Center ‐ Università degli Studi di Milano ‐ IRCCS Istituto Auxologico Italiano, Milano, Italy
| | - Dorothée Lulé
- Department of Neurology ‐ University of Ulm, Ulm, Germany
| | | | - Vincenzo Silani
- Department of Neurology and Laboratory of Neuroscience ‐ “Dino Ferrari” Center ‐ Università degli Studi di Milano ‐ IRCCS Istituto Auxologico Italiano, Milano, Italy
| | - Giuseppe Riva
- Applied Technology for Neuro‐Psychology Lab, IRCCS Istituto Auxologico Italiano, Milan, Italy
| |
Collapse
|
46
|
Pires G, Nunes U, Castelo-Branco M. Comparison of a row-column speller vs. a novel lateral single-character speller: Assessment of BCI for severe motor disabled patients. Clin Neurophysiol 2012; 123:1168-81. [DOI: 10.1016/j.clinph.2011.10.040] [Citation(s) in RCA: 75] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2011] [Revised: 10/17/2011] [Accepted: 10/19/2011] [Indexed: 11/17/2022]
|
47
|
Jin J, Allison BZ, Wang X, Neuper C. A combined brain–computer interface based on P300 potentials and motion-onset visual evoked potentials. J Neurosci Methods 2012; 205:265-76. [DOI: 10.1016/j.jneumeth.2012.01.004] [Citation(s) in RCA: 45] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2011] [Revised: 12/27/2011] [Accepted: 01/09/2012] [Indexed: 10/14/2022]
|
48
|
Targeting an efficient target-to-target interval for P300 speller brain-computer interfaces. Med Biol Eng Comput 2012; 50:289-96. [PMID: 22350331 DOI: 10.1007/s11517-012-0868-x] [Citation(s) in RCA: 29] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2011] [Accepted: 02/06/2012] [Indexed: 10/28/2022]
Abstract
Longer target-to-target intervals (TTI) produce greater P300 event-related potential amplitude, which can increase brain-computer interface (BCI) classification accuracy and decrease the number of flashes needed for accurate character classification. However, longer TTIs requires more time for each trial, which will decrease the information transfer rate of BCI. In this paper, a P300 BCI using a 7 × 12 matrix explored new flash patterns (16-, 18- and 21-flash pattern) with different TTIs to assess the effects of TTI on P300 BCI performance. The new flash patterns were designed to minimize TTI, decrease repetition blindness, and examine the temporal relationship between each flash of a given stimulus by placing a minimum of one (16-flash pattern), two (18-flash pattern), or three (21-flash pattern) non-target flashes between each target flashes. Online results showed that the 16-flash pattern yielded the lowest classification accuracy among the three patterns. The results also showed that the 18-flash pattern provides a significantly higher information transfer rate (ITR) than the 21-flash pattern; both patterns provide high ITR and high accuracy for all subjects.
Collapse
|
49
|
Brain computer interfaces, a review. SENSORS 2012; 12:1211-79. [PMID: 22438708 PMCID: PMC3304110 DOI: 10.3390/s120201211] [Citation(s) in RCA: 706] [Impact Index Per Article: 58.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/29/2011] [Revised: 01/16/2012] [Accepted: 01/29/2012] [Indexed: 11/16/2022]
Abstract
A brain-computer interface (BCI) is a hardware and software communications system that permits cerebral activity alone to control computers or external devices. The immediate goal of BCI research is to provide communications capabilities to severely disabled people who are totally paralyzed or 'locked in' by neurological neuromuscular disorders, such as amyotrophic lateral sclerosis, brain stem stroke, or spinal cord injury. Here, we review the state-of-the-art of BCIs, looking at the different steps that form a standard BCI: signal acquisition, preprocessing or signal enhancement, feature extraction, classification and the control interface. We discuss their advantages, drawbacks, and latest advances, and we survey the numerous technologies reported in the scientific literature to design each step of a BCI. First, the review examines the neuroimaging modalities used in the signal acquisition step, each of which monitors a different functional brain activity such as electrical, magnetic or metabolic activity. Second, the review discusses different electrophysiological control signals that determine user intentions, which can be detected in brain activity. Third, the review includes some techniques used in the signal enhancement step to deal with the artifacts in the control signals and improve the performance. Fourth, the review studies some mathematic algorithms used in the feature extraction and classification steps which translate the information in the control signals into commands that operate a computer or other device. Finally, the review provides an overview of various BCI applications that control a range of devices.
Collapse
|
50
|
Geuze J, Farquhar JDR, Desain P. Dense codes at high speeds: varying stimulus properties to improve visual speller performance. J Neural Eng 2012; 9:016009. [PMID: 22248483 DOI: 10.1088/1741-2560/9/1/016009] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|