1
|
Iturrate I, Chavarriaga R, Millán JDR. General principles of machine learning for brain-computer interfacing. HANDBOOK OF CLINICAL NEUROLOGY 2020; 168:311-328. [PMID: 32164862 DOI: 10.1016/b978-0-444-63934-9.00023-8] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
Abstract
Brain-computer interfaces (BCIs) are systems that translate brain activity patterns into commands that can be executed by an artificial device. This enables the possibility of controlling devices such as a prosthetic arm or exoskeleton, a wheelchair, typewriting applications, or games directly by modulating our brain activity. For this purpose, BCI systems rely on signal processing and machine learning algorithms to decode the brain activity. This chapter provides an overview of the main steps required to do such a process, including signal preprocessing, feature extraction and selection, and decoding. Given the large amount of possible methods that can be used for these processes, a comprehensive review of them is beyond the scope of this chapter, and it is focused instead on the general principles that should be taken into account, as well as discussing good practices on how these methods should be applied and evaluated for proper design of reliable BCI systems.
Collapse
Affiliation(s)
- Iñaki Iturrate
- Center for Neuroprosthetics, École Polytechnique Fédérale de Lausanne, Geneva, Switzerland
| | - Ricardo Chavarriaga
- Center for Neuroprosthetics, École Polytechnique Fédérale de Lausanne, Geneva, Switzerland; Institute of Applied Information Technology (InIT), Zurich University of Applied Sciences ZHAW, Winterthur, Switzerland.
| | - José Del R Millán
- Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, United States; Department of Neurology, The University of Texas at Austin, Austin, TX, United States
| |
Collapse
|
2
|
Isaksen JL, Mohebbi A, Puthusserypady S. Optimal pseudorandom sequence selection for online c-VEP based BCI control applications. PLoS One 2017; 12:e0184785. [PMID: 28902895 PMCID: PMC5597237 DOI: 10.1371/journal.pone.0184785] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2016] [Accepted: 08/30/2017] [Indexed: 11/19/2022] Open
Abstract
Background In a c-VEP BCI setting, test subjects can have highly varying performances when different pseudorandom sequences are applied as stimulus, and ideally, multiple codes should be supported. On the other hand, repeating the experiment with many different pseudorandom sequences is a laborious process. Aims This study aimed to suggest an efficient method for choosing the optimal stimulus sequence based on a fast test and simple measures to increase the performance and minimize the time consumption for research trials. Methods A total of 21 healthy subjects were included in an online wheelchair control task and completed the same task using stimuli based on the m-code, the gold-code, and the Barker-code. Correct/incorrect identification and time consumption were obtained for each identification. Subject-specific templates were characterized and used in a forward-step first-order model to predict the chance of completion and accuracy score. Results No specific pseudorandom sequence showed superior accuracy on the group basis. When isolating the individual performances with the highest accuracy, time consumption per identification was not significantly increased. The Accuracy Score aids in predicting what pseudorandom sequence will lead to the best performance using only the templates. The Accuracy Score was higher when the template resembled a delta function the most and when repeated templates were consistent. For completion prediction, only the shape of the template was a significant predictor. Conclusions The simple and fast method presented in this study as the Accuracy Score, allows c-VEP based BCI systems to support multiple pseudorandom sequences without increase in trial length. This allows for more personalized BCI systems with better performance to be tested without increased costs.
Collapse
Affiliation(s)
- Jonas L. Isaksen
- Department of Electrical Engineering, Technical University of Denmark, Kongens Lyngby, Denmark
- Laboratory of Experimental Cardiology, Department of Biomedical Sciences, University of Copenhagen, Copenhagen, Denmark
- * E-mail:
| | - Ali Mohebbi
- Department of Electrical Engineering, Technical University of Denmark, Kongens Lyngby, Denmark
| | | |
Collapse
|
3
|
Chavarriaga R, Fried-Oken M, Kleih S, Lotte F, Scherer R. Heading for new shores! Overcoming pitfalls in BCI design. BRAIN-COMPUTER INTERFACES 2016; 4:60-73. [PMID: 29629393 DOI: 10.1080/2326263x.2016.1263916] [Citation(s) in RCA: 53] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Abstract
Research in brain-computer interfaces has achieved impressive progress towards implementing assistive technologies for restoration or substitution of lost motor capabilities, as well as supporting technologies for able-bodied subjects. Notwithstanding this progress, effective translation of these interfaces from proof-of concept prototypes into reliable applications remains elusive. As a matter of fact, most of the current BCI systems cannot be used independently for long periods of time by their intended end-users. Multiple factors that impair achieving this goal have already been identified. However, it is not clear how do they affect the overall BCI performance or how they should be tackled. This is worsened by the publication bias where only positive results are disseminated, preventing the research community from learning from its errors. This paper is the result of a workshop held at the 6th International BCI meeting in Asilomar. We summarize here the discussion on concrete research avenues and guidelines that may help overcoming common pitfalls and make BCIs become a useful alternative communication device.
Collapse
Affiliation(s)
- Ricardo Chavarriaga
- Defitech Chair in Brain-Machine Interface (CNBI), Center for Neuroprosthetics, Ecole Polytechnique Fédérale de Lausanne, Switzerland
| | - Melanie Fried-Oken
- Oregon Health & Science University, Institute on Development and Disability, Portland, Oregon USA
| | - Sonja Kleih
- Institute of Psychology, University of Würzburg, Marcusstraße 9-11, Würzburg, 97070, Germany
| | - Fabien Lotte
- Inria Bordeaux Sud-Ouest/LaBRI, 200 avenue de la vieille tour, 33405, Talence cedex, France
| | - Reinhold Scherer
- Institute of Neural Engineering, BCI-Lab, Graz University of Technology, Stremayrgasse 16/IV, 8010 Graz, Austria
| |
Collapse
|
4
|
Chavarriaga R, Sobolewski A, Millán JDR. Errare machinale est: the use of error-related potentials in brain-machine interfaces. Front Neurosci 2014; 8:208. [PMID: 25100937 PMCID: PMC4106211 DOI: 10.3389/fnins.2014.00208] [Citation(s) in RCA: 139] [Impact Index Per Article: 13.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2014] [Accepted: 06/30/2014] [Indexed: 11/13/2022] Open
Abstract
The ability to recognize errors is crucial for efficient behavior. Numerous studies have identified electrophysiological correlates of error recognition in the human brain (error-related potentials, ErrPs). Consequently, it has been proposed to use these signals to improve human-computer interaction (HCI) or brain-machine interfacing (BMI). Here, we present a review of over a decade of developments toward this goal. This body of work provides consistent evidence that ErrPs can be successfully detected on a single-trial basis, and that they can be effectively used in both HCI and BMI applications. We first describe the ErrP phenomenon and follow up with an analysis of different strategies to increase the robustness of a system by incorporating single-trial ErrP recognition, either by correcting the machine's actions or by providing means for its error-based adaptation. These approaches can be applied both when the user employs traditional HCI input devices or in combination with another BMI channel. Finally, we discuss the current challenges that have to be overcome in order to fully integrate ErrPs into practical applications. This includes, in particular, the characterization of such signals during real(istic) applications, as well as the possibility of extracting richer information from them, going beyond the time-locked decoding that dominates current approaches.
Collapse
Affiliation(s)
- Ricardo Chavarriaga
- Defitech Chair in Non-Invasive Brain-Machine Interface, Center for Neuroprosthetics, School of Engineering, Ecole Polytechnique Fédérale de Lausanne Lausanne, Switzerland
| | - Aleksander Sobolewski
- Defitech Chair in Non-Invasive Brain-Machine Interface, Center for Neuroprosthetics, School of Engineering, Ecole Polytechnique Fédérale de Lausanne Lausanne, Switzerland
| | - José Del R Millán
- Defitech Chair in Non-Invasive Brain-Machine Interface, Center for Neuroprosthetics, School of Engineering, Ecole Polytechnique Fédérale de Lausanne Lausanne, Switzerland
| |
Collapse
|
5
|
Schettini F, Aloise F, Aricò P, Salinari S, Mattia D, Cincotti F. Self-calibration algorithm in an asynchronous P300-based brain-computer interface. J Neural Eng 2014; 11:035004. [PMID: 24838347 DOI: 10.1088/1741-2560/11/3/035004] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE Reliability is a desirable characteristic of brain-computer interface (BCI) systems when they are intended to be used under non-experimental operating conditions. In addition, their overall usability is influenced by the complex and frequent procedures that are required for configuration and calibration. Earlier studies examined the issue of asynchronous control in P300-based BCIs, introducing dynamic stopping and automatic control suspension features. This report proposes and evaluates an algorithm for the automatic recalibration of the classifier's parameters using unsupervised data. APPROACH Ten healthy subjects participated in five P300-based BCI sessions throughout a single day. First, we examined whether continuous adaptation of control parameters improved the accuracy of the asynchronous system over time. Then, we assessed the performance of the self-calibration algorithm with respect to the no-recalibration and supervised calibration conditions with regard to system accuracy and communication efficiency. MAIN RESULTS Offline tests demonstrated that continuous adaptation of the control parameters significantly increased the communication efficiency of asynchronous P300-based BCIs. The self-calibration algorithm correctly assigned labels to unsupervised data with 95% accuracy, effecting communication efficiency that was comparable with that of supervised repeated calibration. SIGNIFICANCE Although additional online tests that involve end-users under non-experimental conditions are needed, these preliminary results are encouraging, from which we conclude that the self-calibration algorithm is a promising solution to improve P300-based BCI usability and reliability.
Collapse
Affiliation(s)
- F Schettini
- Neuroelectrical Imaging and BCI Lab, IRCCS Fondazione Santa Lucia, Rome, Italy. Department of Computer, Control, and Management Engineering, University of Rome 'Sapienza', Rome, Italy
| | | | | | | | | | | |
Collapse
|
6
|
Thompson DE, Quitadamo LR, Mainardi L, Laghari KUR, Gao S, Kindermans PJ, Simeral JD, Fazel-Rezai R, Matteucci M, Falk TH, Bianchi L, Chestek CA, Huggins JE. Performance measurement for brain-computer or brain-machine interfaces: a tutorial. J Neural Eng 2014; 11:035001. [PMID: 24838070 DOI: 10.1088/1741-2560/11/3/035001] [Citation(s) in RCA: 52] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/08/2023]
Abstract
OBJECTIVE Brain-computer interfaces (BCIs) have the potential to be valuable clinical tools. However, the varied nature of BCIs, combined with the large number of laboratories participating in BCI research, makes uniform performance reporting difficult. To address this situation, we present a tutorial on performance measurement in BCI research. APPROACH A workshop on this topic was held at the 2013 International BCI Meeting at Asilomar Conference Center in Pacific Grove, California. This paper contains the consensus opinion of the workshop members, refined through discussion in the following months and the input of authors who were unable to attend the workshop. MAIN RESULTS Checklists for methods reporting were developed for both discrete and continuous BCIs. Relevant metrics are reviewed for different types of BCI research, with notes on their use to encourage uniform application between laboratories. SIGNIFICANCE Graduate students and other researchers new to BCI research may find this tutorial a helpful introduction to performance measurement in the field.
Collapse
Affiliation(s)
- David E Thompson
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA
| | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
7
|
Huggins JE, Guger C, Allison B, Anderson CW, Batista A, Brouwer AM(AM, Brunner C, Chavarriaga R, Fried-Oken M, Gunduz A, Gupta D, Kübler A, Leeb R, Lotte F, Miller LE, Müller-Putz G, Rutkowski T, Tangermann M, Thompson DE. Workshops of the Fifth International Brain-Computer Interface Meeting: Defining the Future. BRAIN-COMPUTER INTERFACES 2014; 1:27-49. [PMID: 25485284 PMCID: PMC4255956 DOI: 10.1080/2326263x.2013.876724] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/05/2023]
Abstract
The Fifth International Brain-Computer Interface (BCI) Meeting met June 3-7th, 2013 at the Asilomar Conference Grounds, Pacific Grove, California. The conference included 19 workshops covering topics in brain-computer interface and brain-machine interface research. Topics included translation of BCIs into clinical use, standardization and certification, types of brain activity to use for BCI, recording methods, the effects of plasticity, special interest topics in BCIs applications, and future BCI directions. BCI research is well established and transitioning to practical use to benefit people with physical impairments. At the same time, new applications are being explored, both for people with physical impairments and beyond. Here we provide summaries of each workshop, illustrating the breadth and depth of BCI research and high-lighting important issues for future research and development.
Collapse
Affiliation(s)
- Jane E. Huggins
- Department of Physical Medicine and Rehabilitation, Department of Biomedical Engineering, University of Michigan, Ann Arbor, Michigan, United States, 325 East Eisenhower, Room 3017; Ann Arbor, Michigan 48108-5744, 734-936-7177
| | - Christoph Guger
- Christoph Guger, g.tec medical engineering GmbH/Guger Technologies OG, Austria, Sierningstrasse 14, 4521 Schiedlberg, Austria, +43725122240-0
| | - Brendan Allison
- University of California at San Diego, La Jolla, CA 91942 (415) 490 7551
| | - Charles W. Anderson
- Department of Computer Science, Colorado State University, Fort Collins, CO 80523; telephone: 970-491-7491
| | - Aaron Batista
- Department of Bioengineering, Swanson School of Engineering, University of Pittsburgh, 3501 5th Av, BST3 4074; Pittsburgh, PA 15261; (412) 383-5394
| | - Anne-Marie (A.-M.) Brouwer
- The Netherlands Organization for Applied Scientific Research; P.O. Box 23/Kampweg 5, 3769 ZG Soesterberg, the Netherlands, ++31 (0)888 665960
| | - Clemens Brunner
- Institute for Knowledge Discovery, Laboratory of Brain-Computer Interfaces, Graz University of Technology, Inffeldgasse 13/4, 8010; Graz, Austria
| | - Ricardo Chavarriaga
- Center for Neuroprosthetics, École Polytechnique Fédérale de Lausanne, Switzerland, EPFL-STI-CNBI, Station 11, 1005 Lausanne, Switzerland; Telephone: +41 21 693 6968
| | - Melanie Fried-Oken
- Oregon Health & Science University; Institute on Development & Disability; 707 SW Gaines Street; Portland, Oregon, United States; O: 503.494.7587, F: 503.494.6868
| | - Aysegul Gunduz
- Department of Biomedical Engineering, University of Florida, Gainesville, FL 32611, USA; Phone: +1 (352) 273 6877; Fax: +1 (352) 273 9221
| | - Disha Gupta
- Dept. of Neurology, Albany Medical College/Brain Computer Interfacing Lab, Wadsworth Center, NY State Dept. of Health, Albany, New York, USA
| | - Andrea Kübler
- Institute of Psychology, University of Würzburg; Marcusstr.9-11; 97070 Würzburg, Germany. Phone.: 0049 931 31 80179; Fax: 0049 931 31 82424
| | - Robert Leeb
- Center for Neuroprosthetics, École Polytechnique Fédérale de Lausanne, Switzerland
| | - Fabien Lotte
- Inria Bordeaux Sud-Ouest/LaBRI, 200 avenue de la vieille tour, 33405, Talence Cedex, France, Tel: +33 5 24 57 41 26
| | - Lee E. Miller
- Departments of Physiology, Physical Medicine and Rehab, and Biomedical Engineering; Feinberg School of Medicine; Northwestern University; Chicago, Illinois, United States; Ward 5-01; 303 East Chicago Avenue; Chicago, Illinois 60611; Phone: (312) 503 – 8677; Fax: (312) 503 – 5101
| | - Gernot Müller-Putz
- Institute for Knowledge Discovery, Laboratory of Brain-Computer Interfaces, Graz University of Technology, Inffeldgasse 13/4, 8010; Graz, Austria
| | - Tomasz Rutkowski
- Life Science Center of TARA, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki, 305-8577 Japan; TEL: +81 (0)29-853-6261
| | - Michael Tangermann
- Excellence Cluster BrainLinks-BrainTools, Dept. Computer Science, University of Freiburg, Freiburg, Germany, Albertstr. 23; 79104 Freiburg; Germany; Phone: +49.(0)761.2038423, Fax : +49.(0)761.2038417
| | - David Edward Thompson
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, Michigan, United States, 2800 Plymouth Road, Bdlg 26 Rm G06W-B; Ann Arbor, MI 48109; 734-763-7104
| |
Collapse
|
8
|
Aloise F, Aricò P, Schettini F, Salinari S, Mattia D, Cincotti F. Asynchronous gaze-independent event-related potential-based brain-computer interface. Artif Intell Med 2013; 59:61-9. [PMID: 24080078 DOI: 10.1016/j.artmed.2013.07.006] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2012] [Revised: 07/30/2013] [Accepted: 07/31/2013] [Indexed: 10/26/2022]
Abstract
OBJECTIVE In this study a gaze independent event related potential (ERP)-based brain computer interface (BCI) for communication purpose was combined with an asynchronous classifier endowed with dynamical stopping feature. The aim was to evaluate if and how the performance of such asynchronous system could be negatively affected in terms of communication efficiency and robustness to false positives during the intentional no-control state. MATERIAL AND METHODS The proposed system was validated with the participation of 9 healthy subjects. A comparison was performed between asynchronous and synchronous classification technique outputs while users were controlling the same gaze independent BCI interface. The performance of both classification techniques were assessed both off-line and on-line by means of the efficiency metric introduced by Bianchi et al. (2007). This latter metric allows to set a different misclassification cost for wrong classifications and abstentions. Robustness was evaluated as the rate of false positives occurring during voluntary no-control states. RESULTS The asynchronous classifier did not exhibited significantly higher accuracy or lower error rate with respect to the synchronous classifier (accuracy: 74.66% versus 87.96%, error rate: 7.11% versus 12.04% respectively). However, the on-line and off-line analysis revealed that the communication efficiency was significantly improved (p<.05) with the asynchronous classification modality as compared with the synchronous. Furthermore, the asynchronous classifier proved to be robust to false positives during intentional no-control state which occur during the ongoing visual stimulation (less than 1 false positive every 6min). CONCLUSION As such, the proposed ERP-BCI system which combines an asynchronous classifier with a gaze independent interface is a promising solution to be further explored in order to increase the general usability of ERP-based BCI systems designed for severely disabled people with an impairment of the voluntary control of eye movements. In fact, the asynchronous classifier can improve communication efficiency automatically adapting the number of stimulus repetitions to the current user's state and suspending the control if he/she does not intend to select an item.
Collapse
Affiliation(s)
- Fabio Aloise
- Neuroelectrical Imaging and BCI Lab, Fondazione Santa Lucia IRCCS, Via Ardeatina 306, 00142 Rome, Italy; Department of Computer, Control, and Management Engineering Antonio Ruberti, Sapienza University, Via Ariosto 25, 00185 Rome, Italy.
| | | | | | | | | | | |
Collapse
|
9
|
A new dual-frequency stimulation method to increase the number of visual stimuli for multi-class SSVEP-based brain–computer interface (BCI). Brain Res 2013; 1515:66-77. [DOI: 10.1016/j.brainres.2013.03.050] [Citation(s) in RCA: 62] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2012] [Revised: 03/01/2013] [Accepted: 03/26/2013] [Indexed: 11/23/2022]
|
10
|
Thomas E, Dyson M, Clerc M. An analysis of performance evaluation for motor-imagery based BCI. J Neural Eng 2013; 10:031001. [PMID: 23639955 DOI: 10.1088/1741-2560/10/3/031001] [Citation(s) in RCA: 62] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
In recent years, numerous brain-computer interfaces (BCIs) based on motor-imagery have been proposed which incorporate features such as adaptive classification, error detection and correction, fusion with auxiliary signals and shared control capabilities. Due to the added complexity of such algorithms, the evaluation strategy and metrics used for analysis must be carefully chosen to accurately represent the performance of the BCI. In this article, metrics are reviewed and contrasted using both simulated examples and experimental data. Furthermore, a review of the recent literature is presented to determine how BCIs are evaluated, in particular, focusing on the relationship between how the data are used relative to the BCI subcomponent under investigation. From the analysis performed in this study, valuable guidelines are presented regarding the choice of metrics and evaluation strategy dependent upon any chosen BCI paradigm.
Collapse
Affiliation(s)
- Eoin Thomas
- INRIA, 2004, Route des Lucioles, F-06902 Sophia Antipolis, France.
| | | | | |
Collapse
|
11
|
Yuan P, Gao X, Allison B, Wang Y, Bin G, Gao S. A study of the existing problems of estimating the information transfer rate in online brain–computer interfaces. J Neural Eng 2013; 10:026014. [PMID: 23448963 DOI: 10.1088/1741-2560/10/2/026014] [Citation(s) in RCA: 91] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
|
12
|
Targeting an efficient target-to-target interval for P300 speller brain-computer interfaces. Med Biol Eng Comput 2012; 50:289-96. [PMID: 22350331 DOI: 10.1007/s11517-012-0868-x] [Citation(s) in RCA: 29] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2011] [Accepted: 02/06/2012] [Indexed: 10/28/2022]
Abstract
Longer target-to-target intervals (TTI) produce greater P300 event-related potential amplitude, which can increase brain-computer interface (BCI) classification accuracy and decrease the number of flashes needed for accurate character classification. However, longer TTIs requires more time for each trial, which will decrease the information transfer rate of BCI. In this paper, a P300 BCI using a 7 × 12 matrix explored new flash patterns (16-, 18- and 21-flash pattern) with different TTIs to assess the effects of TTI on P300 BCI performance. The new flash patterns were designed to minimize TTI, decrease repetition blindness, and examine the temporal relationship between each flash of a given stimulus by placing a minimum of one (16-flash pattern), two (18-flash pattern), or three (21-flash pattern) non-target flashes between each target flashes. Online results showed that the 16-flash pattern yielded the lowest classification accuracy among the three patterns. The results also showed that the 18-flash pattern provides a significantly higher information transfer rate (ITR) than the 21-flash pattern; both patterns provide high ITR and high accuracy for all subjects.
Collapse
|