1
|
Kojima S, Kanoh S. An auditory brain-computer interface based on selective attention to multiple tone streams. PLoS One 2024; 19:e0303565. [PMID: 38781127 PMCID: PMC11115270 DOI: 10.1371/journal.pone.0303565] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Accepted: 04/27/2024] [Indexed: 05/25/2024] Open
Abstract
In this study, we attempted to improve brain-computer interface (BCI) systems by means of auditory stream segregation in which alternately presented tones are perceived as sequences of various different tones (streams). A 3-class BCI using three tone sequences, which were perceived as three different tone streams, was investigated and evaluated. Each presented musical tone was generated by a software synthesizer. Eleven subjects took part in the experiment. Stimuli were presented to each user's right ear. Subjects were requested to attend to one of three streams and to count the number of target stimuli in the attended stream. In addition, 64-channel electroencephalogram (EEG) and two-channel electrooculogram (EOG) signals were recorded from participants with a sampling frequency of 1000 Hz. The measured EEG data were classified based on Riemannian geometry to detect the object of the subject's selective attention. P300 activity was elicited by the target stimuli in the segregated tone streams. In five out of eleven subjects, P300 activity was elicited only by the target stimuli included in the attended stream. In a 10-fold cross validation test, a classification accuracy over 80% for five subjects and over 75% for nine subjects was achieved. For subjects whose accuracy was lower than 75%, either the P300 was also elicited for nonattended streams or the amplitude of P300 was small. It was concluded that the number of selected BCI systems based on auditory stream segregation can be increased to three classes, and these classes can be detected by a single ear without the aid of any visual modality.
Collapse
Affiliation(s)
- Simon Kojima
- Graduate School of Engineering and Science, Shibaura Institute of Technology, Koto-ku, Tokyo, Japan
| | - Shin’ichiro Kanoh
- Graduate School of Engineering and Science, Shibaura Institute of Technology, Koto-ku, Tokyo, Japan
- College of Engineering, Shibaura Institute of Technology, Koto-ku, Tokyo, Japan
| |
Collapse
|
2
|
Keough JR, Irvine B, Kelly D, Wrightson J, Comaduran Marquez D, Kinney-Lang E, Kirton A. Fatigue in children using motor imagery and P300 brain-computer interfaces. J Neuroeng Rehabil 2024; 21:61. [PMID: 38658998 PMCID: PMC11040843 DOI: 10.1186/s12984-024-01349-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2023] [Accepted: 03/25/2024] [Indexed: 04/26/2024] Open
Abstract
BACKGROUND Brain-computer interface (BCI) technology offers children with quadriplegic cerebral palsy unique opportunities for communication, environmental exploration, learning, and game play. Research in adults demonstrates a negative impact of fatigue on BCI enjoyment, while effects on BCI performance are variable. To date, there have been no pediatric studies of BCI fatigue. The purpose of this study was to assess the effects of two different BCI paradigms, motor imagery and visual P300, on the development of self-reported fatigue and an electroencephalography (EEG) biomarker of fatigue in typically developing children. METHODS Thirty-seven typically-developing school-aged children were recruited to a prospective, crossover study. Participants attended three sessions: (A) motor imagery-BCI, (B) visual P300-BCI, and (C) video viewing (control). The motor imagery task involved an imagined left- or right-hand squeeze. The P300 task involved attending to one square on a 3 × 3 grid during a random single flash sequence. Each paradigm had respective calibration periods and a similar visual counting game. Primary outcomes were self-reported fatigue and the power of the EEG alpha band both collected during resting-state periods pre- and post-task. Self-reported fatigue was measured using a 10-point visual analog scale. EEG alpha band power was calculated as the integrated power spectral density from 8 to 12 Hz of the EEG spectrum. RESULTS Thirty-two children completed the protocol (age range 7-16, 63% female). Self-reported fatigue and EEG alpha band power increased across all sessions (F(1,155) = 33.9, p < 0.001; F = 5.0(1,149), p = 0.027 respectively). No differences in fatigue development were observed between session types. There was no correlation between self-reported fatigue and EEG alpha band power change. BCI performance varied between participants and paradigms as expected but was not associated with self-reported fatigue or EEG alpha band power. CONCLUSION Short periods (30-mintues) of BCI use can increase self-reported fatigue and EEG alpha band power to a similar degree in children performing motor imagery and P300 BCI paradigms. Performance was not associated with our measures of fatigue; the impact of fatigue on useability and enjoyment is unclear. Our results reflect the variability of fatigue and the BCI experience more broadly in children and warrant further investigation.
Collapse
Affiliation(s)
- Joanna Rg Keough
- Departments of Pediatrics and Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Brian Irvine
- Departments of Pediatrics and Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Dion Kelly
- Departments of Pediatrics and Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - James Wrightson
- Departments of Pediatrics and Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
- Faculty of Medicine, University of British Columbia, Vancouver, BC, Canada
| | - Daniel Comaduran Marquez
- Departments of Pediatrics and Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Eli Kinney-Lang
- Departments of Pediatrics and Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Adam Kirton
- Departments of Pediatrics and Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.
| |
Collapse
|
3
|
Séguin P, Maby E, Fouillen M, Otman A, Luauté J, Giraux P, Morlet D, Mattout J. The challenge of controlling an auditory BCI in the case of severe motor disability. J Neuroeng Rehabil 2024; 21:9. [PMID: 38238759 PMCID: PMC10795353 DOI: 10.1186/s12984-023-01289-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Accepted: 11/29/2023] [Indexed: 01/22/2024] Open
Abstract
BACKGROUND The locked-in syndrome (LIS), due to a lesion in the pons, impedes communication. This situation can also be met after some severe brain injury or in advanced Amyotrophic Lateral Sclerosis (ALS). In the most severe condition, the persons cannot communicate at all because of a complete oculomotor paralysis (Complete LIS or CLIS). This even prevents the detection of consciousness. Some studies suggest that auditory brain-computer interface (BCI) could restore a communication through a « yes-no» code. METHODS We developed an auditory EEG-based interface which makes use of voluntary modulations of attention, to restore a yes-no communication code in non-responding persons. This binary BCI uses repeated speech sounds (alternating "yes" on the right ear and "no" on the left ear) corresponding to either frequent (short) or rare (long) stimuli. Users are instructed to pay attention to the relevant stimuli only. We tested this BCI with 18 healthy subjects, and 7 people with severe motor disability (3 "classical" persons with locked-in syndrome and 4 persons with ALS). RESULTS We report online BCI performance and offline event-related potential analysis. On average in healthy subjects, online BCI accuracy reached 86% based on 50 questions. Only one out of 18 subjects could not perform above chance level. Ten subjects had an accuracy above 90%. However, most patients could not produce online performance above chance level, except for two people with ALS who obtained 100% accuracy. We report individual event-related potentials and their modulation by attention. In addition to the classical P3b, we observed a signature of sustained attention on responses to frequent sounds, but in healthy subjects and patients with good BCI control only. CONCLUSIONS Auditory BCI can be very well controlled by healthy subjects, but it is not a guarantee that it can be readily used by the target population of persons in LIS or CLIS. A conclusion that is supported by a few previous findings in BCI and should now trigger research to assess the reasons of such a gap in order to propose new and efficient solutions. CLINICAL TRIAL REGISTRATIONS No. NCT02567201 (2015) and NCT03233282 (2013).
Collapse
Affiliation(s)
- Perrine Séguin
- Lyon Neuroscience Research Center, INSERM UMRS 1028, CNRS UMR 5292, Université Claude Bernard Lyon 1, Université de Lyon, 69000, Lyon, France
- University Hospital of Saint-Etienne, 42000, Saint-Etienne, France
- Jean Monnet University, 42000, Saint-Etienne, France
| | - Emmanuel Maby
- Lyon Neuroscience Research Center, INSERM UMRS 1028, CNRS UMR 5292, Université Claude Bernard Lyon 1, Université de Lyon, 69000, Lyon, France
| | - Mélodie Fouillen
- Lyon Neuroscience Research Center, INSERM UMRS 1028, CNRS UMR 5292, Université Claude Bernard Lyon 1, Université de Lyon, 69000, Lyon, France
| | - Anatole Otman
- Lyon Neuroscience Research Center, INSERM UMRS 1028, CNRS UMR 5292, Université Claude Bernard Lyon 1, Université de Lyon, 69000, Lyon, France
| | - Jacques Luauté
- Lyon Neuroscience Research Center, INSERM UMRS 1028, CNRS UMR 5292, Université Claude Bernard Lyon 1, Université de Lyon, 69000, Lyon, France
- Hospices Civils de Lyon, 69000, Lyon, France
| | - Pascal Giraux
- Lyon Neuroscience Research Center, INSERM UMRS 1028, CNRS UMR 5292, Université Claude Bernard Lyon 1, Université de Lyon, 69000, Lyon, France
- University Hospital of Saint-Etienne, 42000, Saint-Etienne, France
- Jean Monnet University, 42000, Saint-Etienne, France
| | - Dominique Morlet
- Lyon Neuroscience Research Center, INSERM UMRS 1028, CNRS UMR 5292, Université Claude Bernard Lyon 1, Université de Lyon, 69000, Lyon, France
| | - Jérémie Mattout
- Lyon Neuroscience Research Center, INSERM UMRS 1028, CNRS UMR 5292, Université Claude Bernard Lyon 1, Université de Lyon, 69000, Lyon, France.
| |
Collapse
|
4
|
Canny E, Vansteensel MJ, van der Salm SMA, Müller-Putz GR, Berezutskaya J. Boosting brain-computer interfaces with functional electrical stimulation: potential applications in people with locked-in syndrome. J Neuroeng Rehabil 2023; 20:157. [PMID: 37980536 PMCID: PMC10656959 DOI: 10.1186/s12984-023-01272-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Accepted: 10/23/2023] [Indexed: 11/20/2023] Open
Abstract
Individuals with a locked-in state live with severe whole-body paralysis that limits their ability to communicate with family and loved ones. Recent advances in brain-computer interface (BCI) technology have presented a potential alternative for these people to communicate by detecting neural activity associated with attempted hand or speech movements and translating the decoded intended movements to a control signal for a computer. A technique that could potentially enrich the communication capacity of BCIs is functional electrical stimulation (FES) of paralyzed limbs and face to restore body and facial movements of paralyzed individuals, allowing to add body language and facial expression to communication BCI utterances. Here, we review the current state of the art of existing BCI and FES work in people with paralysis of body and face and propose that a combined BCI-FES approach, which has already proved successful in several applications in stroke and spinal cord injury, can provide a novel promising mode of communication for locked-in individuals.
Collapse
Affiliation(s)
- Evan Canny
- Department of Neurology and Neurosurgery, Brain Center, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Mariska J Vansteensel
- Department of Neurology and Neurosurgery, Brain Center, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Sandra M A van der Salm
- Department of Neurology and Neurosurgery, Brain Center, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Gernot R Müller-Putz
- Institute of Neural Engineering, Laboratory of Brain-Computer Interfaces, Graz University of Technology, Graz, Austria
| | - Julia Berezutskaya
- Department of Neurology and Neurosurgery, Brain Center, University Medical Center Utrecht, Utrecht, The Netherlands.
| |
Collapse
|
5
|
Van de Wauw C, Riecke L, Goebel R, Kaas A, Sorger B. Talking with hands and feet: Selective somatosensory attention and fMRI enable robust and convenient brain-based communication. Neuroimage 2023; 276:120172. [PMID: 37230207 DOI: 10.1016/j.neuroimage.2023.120172] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2022] [Revised: 03/07/2023] [Accepted: 05/15/2023] [Indexed: 05/27/2023] Open
Abstract
In brain-based communication, voluntarily modulated brain signals (instead of motor output) are utilized to interact with the outside world. The possibility to circumvent the motor system constitutes an important alternative option for severely paralyzed. Most communication brain-computer interface (BCI) paradigms require intact visual capabilities and impose a high cognitive load, but for some patients, these requirements are not given. In these situations, a better-suited, less cognitively demanding information-encoding approach may exploit auditorily-cued selective somatosensory attention to vibrotactile stimulation. Here, we propose, validate and optimize a novel communication-BCI paradigm using differential fMRI activation patterns evoked by selective somatosensory attention to tactile stimulation of the right hand or left foot. Using cytoarchitectonic probability maps and multi-voxel pattern analysis (MVPA), we show that the locus of selective somatosensory attention can be decoded from fMRI-signal patterns in (especially primary) somatosensory cortex with high accuracy and reliability, with the highest classification accuracy (85.93%) achieved when using Brodmann area 2 (SI-BA2) at a probability level of 0.2. Based on this outcome, we developed and validated a novel somatosensory attention-based yes/no communication procedure and demonstrated its high effectiveness even when using only a limited amount of (MVPA) training data. For the BCI user, the paradigm is straightforward, eye-independent, and requires only limited cognitive functioning. In addition, it is BCI-operator friendly given its objective and expertise-independent procedure. For these reasons, our novel communication paradigm has high potential for clinical applications.
Collapse
Affiliation(s)
- Cynthia Van de Wauw
- Department of Cognitive Neuroscience, Maastricht University, Maastricht, The Netherlands.
| | - Lars Riecke
- Department of Cognitive Neuroscience, Maastricht University, Maastricht, The Netherlands
| | - Rainer Goebel
- Department of Cognitive Neuroscience, Maastricht University, Maastricht, The Netherlands; Department of Neuroimaging and Neuromodeling, Netherlands Institute for Neuroscience, Amsterdam, The Netherlands
| | - Amanda Kaas
- Department of Cognitive Neuroscience, Maastricht University, Maastricht, The Netherlands
| | - Bettina Sorger
- Department of Cognitive Neuroscience, Maastricht University, Maastricht, The Netherlands
| |
Collapse
|
6
|
Zilio F, Gomez-Pilar J, Chaudhary U, Fogel S, Fomina T, Synofzik M, Schöls L, Cao S, Zhang J, Huang Z, Birbaumer N, Northoff G. Altered brain dynamics index levels of arousal in complete locked-in syndrome. Commun Biol 2023; 6:757. [PMID: 37474587 PMCID: PMC10359418 DOI: 10.1038/s42003-023-05109-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2022] [Accepted: 07/06/2023] [Indexed: 07/22/2023] Open
Abstract
Complete locked-in syndrome (CLIS) resulting from late-stage amyotrophic lateral sclerosis (ALS) is characterised by loss of motor function and eye movements. The absence of behavioural indicators of consciousness makes the search for neuronal correlates as possible biomarkers clinically and ethically urgent. EEG-based measures of brain dynamics such as power-law exponent (PLE) and Lempel-Ziv complexity (LZC) have been shown to have explanatory power for consciousness and may provide such neuronal indices for patients with CLIS. Here, we validated PLE and LZC (calculated in a dynamic way) as benchmarks of a wide range of arousal states across different reference states of consciousness (e.g., awake, sleep stages, ketamine, sevoflurane). We show a tendency toward high PLE and low LZC, with high intra-subject fluctuations and inter-subject variability in a cohort of CLIS patients with values graded along different arousal states as in our reference data sets. In conclusion, changes in brain dynamics indicate altered arousal in CLIS. Specifically, PLE and LZC are potentially relevant biomarkers to identify or diagnose the arousal level in CLIS and to determine the optimal time point for treatment, including communication attempts.
Collapse
Affiliation(s)
- Federico Zilio
- Department of Philosophy, Sociology, Education and Applied Psychology, University of Padova, Padua, Italy.
| | - Javier Gomez-Pilar
- Biomedical Engineering Group, University of Valladolid, Valladolid, Spain
- Centro de Investigación Biomédica en Red en Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Valladolid, Spain
| | - Ujwal Chaudhary
- BrainPortal Technologies GmbH, Mannheim, Germany
- ALS Voice gGmbH, Mössingen, Germany
| | - Stuart Fogel
- School of Psychology, University of Ottawa, Ottawa, Canada
- Institute of Mental Health Research, University of Ottawa, Ottawa, Canada
| | - Tatiana Fomina
- Department for Empirical Inference, Max Planck Institute for Intelligent Systems, Tübingen, Germany
| | - Matthis Synofzik
- Department of Neurodegenerative Diseases and Hertie-Institute for Clinical Brain Research, University of Tübingen, Tübingen, Germany
- German Center for Neurodegenerative Diseases (DZNE), Tübingen, Germany
| | - Ludger Schöls
- Department of Neurodegenerative Diseases and Hertie-Institute for Clinical Brain Research, University of Tübingen, Tübingen, Germany
- German Center for Neurodegenerative Diseases (DZNE), Tübingen, Germany
| | - Shumei Cao
- Department of Anesthesiology, Fudan University Shanghai Cancer Center, Shanghai, China
| | - Jun Zhang
- Department of Anesthesiology, Fudan University Shanghai Cancer Center, Shanghai, China
| | - Zirui Huang
- Center for Consciousness Science, Department of Anesthesiology, University of Michigan Medical School, Ann Arbor, MI, USA
| | - Niels Birbaumer
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
| | - Georg Northoff
- Institute of Mental Health Research, University of Ottawa, Ottawa, Canada
| |
Collapse
|
7
|
Adama S, Bogdan M. Assessing consciousness in patients with disorders of consciousness using soft-clustering. Brain Inform 2023; 10:16. [PMID: 37450213 PMCID: PMC10348975 DOI: 10.1186/s40708-023-00197-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Accepted: 06/25/2023] [Indexed: 07/18/2023] Open
Abstract
Consciousness is something we experience in our everyday life, more especially between the time we wake up in the morning and go to sleep at night, but also during the rapid eye movement (REM) sleep stage. Disorders of consciousness (DoC) are states in which a person's consciousness is damaged, possibly after a traumatic brain injury. Completely locked-in syndrome (CLIS) patients, on the other hand, display covert states of consciousness. Although they appear unconscious, their cognitive functions are mostly intact. Only, they cannot externally display it due to their quadriplegia and inability to speak. Determining these patients' states constitutes a challenging task. The ultimate goal of the approach presented in this paper is to assess these CLIS patients consciousness states. EEG data from DoC patients are used here first, under the assumption that if the proposed approach is able to accurately assess their consciousness states, it will assuredly do so on CLIS patients too. This method combines different sets of features consisting of spectral, complexity and connectivity measures in order to increase the probability of correctly estimating their consciousness levels. The obtained results showed that the proposed approach was able to correctly estimate several DoC patients' consciousness levels. This estimation is intended as a step prior attempting to communicate with them, in order to maximise the efficiency of brain-computer interfaces (BCI)-based communication systems.
Collapse
Affiliation(s)
- Sophie Adama
- Department of Neuromorphe Information Processing, Leipzig University, Augustusplatz 10, Leipzig, 04109 Germany
| | - Martin Bogdan
- Department of Neuromorphe Information Processing, Leipzig University, Augustusplatz 10, Leipzig, 04109 Germany
| |
Collapse
|
8
|
Ghani U, Jochumsen M, Gyldenvang MB, Niazi IK. Can water-based EEG caps record robust movement-related cortical potentials (MRCPs) for single and multiple joint movements? ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38083438 DOI: 10.1109/embc40787.2023.10340665] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Movement-related cortical potentials (MRCPs) have been used extensively in the literature to develop rehabilitation interventions for people with neurological conditions. In this pilot study, we recorded and extracted MRCPs using a water-based cap to determine whether water-based caps are effective. Five participants took part in the study, where their EEG was recorded during single-joint (dorsiflexion) and multiple-joint (sit-to-stand) lower limb movements. We were able to see clear MRCPs for both movement types with an average peak negativity (PN) latency of +22ms for dorsiflexion and +218ms for sit-to-stand. Similarly, the PN amplitude of -14.89μV was recorded for dorsiflexion and -43.54μV for sit-to-stand. These values were comparable to the values reported in studies using gel-based caps. Based on these results, water-based caps can be an effective way to produce robust MRCPs, which can have many advantages over gel-based caps.Clinical Relevance- The study provides clinicians with a more viable method of collecting EEGs and extracting MRCPs, thus allowing them to design more robust interventions for people with neurological disorders.
Collapse
|
9
|
Norton L, Graham M, Kazazian K, Gofton T, Weijer C, Debicki D, Fernandez-Espejo D, Thenayan EA, Owen AM. Use of functional magnetic resonance imaging to assess cognition and consciousness in severe Guillain-Barré syndrome. Int J Clin Health Psychol 2023; 23:100347. [DOI: 10.1016/j.ijchp.2022.100347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2022] [Accepted: 10/08/2022] [Indexed: 11/13/2022] Open
|
10
|
Savić AM, Novičić M, Ðorđević O, Konstantinović L, Miler-Jerković V. Novel electrotactile brain-computer interface with somatosensory event-related potential based control. Front Hum Neurosci 2023; 17:1096814. [PMID: 37033908 PMCID: PMC10078957 DOI: 10.3389/fnhum.2023.1096814] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2022] [Accepted: 02/28/2023] [Indexed: 04/11/2023] Open
Abstract
Objective A brain computer interface (BCI) allows users to control external devices using non-invasive brain recordings, such as electroencephalography (EEG). We developed and tested a novel electrotactile BCI prototype based on somatosensory event-related potentials (sERP) as control signals, paired with a tactile attention task as a control paradigm. Approach A novel electrotactile BCI comprises commercial EEG device, an electrical stimulator and custom software for EEG recordings, electrical stimulation control, synchronization between devices, signal processing, feature extraction, selection, and classification. We tested a novel BCI control paradigm based on tactile attention on a sensation at a target stimulation location on the forearm. Tactile stimuli were electrical pulses delivered at two proximal locations on the user's forearm for stimulating branches of radial and median nerves, with equal probability of the target and distractor stimuli occurrence, unlike in any other ERP-based BCI design. We proposed a compact electrical stimulation electrodes configuration for delivering electrotactile stimuli (target and distractor) using 2 stimulation channels and 3 stimulation electrodes. We tested the feasibility of a single EEG channel BCI control, to determine pseudo-online BCI performance, in ten healthy subjects. For optimizing the BCI performance we compared the results for two classifiers, sERP averaging approaches, and novel dedicated feature extraction/selection methods via cross-validation procedures. Main results We achieved a single EEG channel BCI classification accuracy in the range of 75.1 to 88.1% for all subjects. We have established an optimal combination of: single trial averaging to obtain sERP, feature extraction/selection methods and classification approach. Significance The obtained results demonstrate that a novel electrotactile BCI paradigm with equal probability of attended (target) and unattended (distractor) stimuli and proximal stimulation sites is feasible. This method may be used to drive restorative BCIs for sensory retraining in stroke or brain injury, or assistive BCIs for communication in severely disabled users.
Collapse
Affiliation(s)
- Andrej M. Savić
- School of Electrical Engineering, University of Belgrade, Belgrade, Serbia
- *Correspondence: Andrej M. Savić,
| | - Marija Novičić
- School of Electrical Engineering, University of Belgrade, Belgrade, Serbia
| | - Olivera Ðorđević
- Faculty of Medicine, University of Belgrade, Belgrade, Serbia
- Clinic for Rehabilitation “Dr. Miroslav Zotović”, Belgrade, Serbia
| | - Ljubica Konstantinović
- Faculty of Medicine, University of Belgrade, Belgrade, Serbia
- Clinic for Rehabilitation “Dr. Miroslav Zotović”, Belgrade, Serbia
| | - Vera Miler-Jerković
- Innovation Center of the School of Electrical Engineering, University of Belgrade, Belgrade, Serbia
| |
Collapse
|
11
|
Adama S, Bogdan M. Application of Soft-Clustering to Assess Consciousness in a CLIS Patient. Brain Sci 2022; 13:brainsci13010065. [PMID: 36672046 PMCID: PMC9856569 DOI: 10.3390/brainsci13010065] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2022] [Revised: 12/12/2022] [Accepted: 12/21/2022] [Indexed: 01/01/2023] Open
Abstract
Completely locked-in (CLIS) patients are characterized by sufficiently intact cognitive functions, but a complete paralysis that prevents them to interact with their surroundings. On one hand, studies have shown that the ability to communicate plays an important part in these patients' quality of life and prognosis. On the other hand, brain-computer interfaces (BCIs) provide a means for them to communicate using their brain signals. However, one major problem for such patients is the difficulty to determine if they are conscious or not at a specific time. This work aims to combine different sets of features consisting of spectral, complexity and connectivity measures, to increase the probability of correctly estimating CLIS patients' consciousness levels. The proposed approach was tested on data from one CLIS patient, which is particular in the sense that the experimenter was able to point out one time frame Δt during which he was undoubtedly conscious. Results showed that the method presented in this paper was able to detect increases and decreases of the patient's consciousness levels. More specifically, increases were observed during this Δt, corroborating the assertion of the experimenter reporting that the patient was definitely conscious then. Assessing the patients' consciousness is intended as a step prior attempting to communicate with them, in order to maximize the efficiency of BCI-based communication systems.
Collapse
|
12
|
Lee HS, Schreiner L, Jo SH, Sieghartsleitner S, Jordan M, Pretl H, Guger C, Park HS. Individual finger movement decoding using a novel ultra-high-density electroencephalography-based brain-computer interface system. Front Neurosci 2022; 16:1009878. [PMID: 36340769 PMCID: PMC9627315 DOI: 10.3389/fnins.2022.1009878] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2022] [Accepted: 10/04/2022] [Indexed: 11/13/2022] Open
Abstract
Brain-Computer Interface (BCI) technology enables users to operate external devices without physical movement. Electroencephalography (EEG) based BCI systems are being actively studied due to their high temporal resolution, convenient usage, and portability. However, fewer studies have been conducted to investigate the impact of high spatial resolution of EEG on decoding precise body motions, such as finger movements, which are essential in activities of daily living. Low spatial sensor resolution, as found in common EEG systems, can be improved by omitting the conventional standard of EEG electrode distribution (the international 10-20 system) and ordinary mounting structures (e.g., flexible caps). In this study, we used newly proposed flexible electrode grids attached directly to the scalp, which provided ultra-high-density EEG (uHD EEG). We explored the performance of the novel system by decoding individual finger movements using a total of 256 channels distributed over the contralateral sensorimotor cortex. Dense distribution and small-sized electrodes result in an inter-electrode distance of 8.6 mm (uHD EEG), while that of conventional EEG is 60 to 65 mm on average. Five healthy subjects participated in the experiment, performed single finger extensions according to a visual cue, and received avatar feedback. This study exploits mu (8-12 Hz) and beta (13-25 Hz) band power features for classification and topography plots. 3D ERD/S activation plots for each frequency band were generated using the MNI-152 template head. A linear support vector machine (SVM) was used for pairwise finger classification. The topography plots showed regular and focal post-cue activation, especially in subjects with optimal signal quality. The average classification accuracy over subjects was 64.8 (6.3)%, with the middle versus ring finger resulting in the highest average accuracy of 70.6 (9.4)%. Further studies are required using the uHD EEG system with real-time feedback and motor imagery tasks to enhance classification performance and establish the basis for BCI finger movement control of external devices.
Collapse
Affiliation(s)
- Hyemin S. Lee
- Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea
| | - Leonhard Schreiner
- g.tec Medical Engineering GmbH, Schiedlberg, Upper Austria, Austria
- Institute for Integrated Circuits, Johannes Kepler University, Linz, Austria
| | - Seong-Hyeon Jo
- Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea
| | | | - Michael Jordan
- g.tec Medical Engineering GmbH, Schiedlberg, Upper Austria, Austria
| | - Harald Pretl
- Institute for Integrated Circuits, Johannes Kepler University, Linz, Austria
| | - Christoph Guger
- g.tec Medical Engineering GmbH, Schiedlberg, Upper Austria, Austria
| | - Hyung-Soon Park
- Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea
| |
Collapse
|
13
|
Spataro R, Xu Y, Xu R, Mandalà G, Allison BZ, Ortner R, Heilinger A, La Bella V, Guger C. How brain-computer interface technology may improve the diagnosis of the disorders of consciousness: A comparative study. Front Neurosci 2022; 16:959339. [PMID: 36033632 PMCID: PMC9404379 DOI: 10.3389/fnins.2022.959339] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2022] [Accepted: 07/18/2022] [Indexed: 01/18/2023] Open
Abstract
Objective Clinical assessment of consciousness relies on behavioural assessments, which have several limitations. Hence, disorder of consciousness (DOC) patients are often misdiagnosed. In this work, we aimed to compare the repetitive assessment of consciousness performed with a clinical behavioural and a Brain-Computer Interface (BCI) approach. Materials and methods For 7 weeks, sixteen DOC patients participated in weekly evaluations using both the Coma Recovery Scale-Revised (CRS-R) and a vibrotactile P300 BCI paradigm. To use the BCI, patients had to perform an active mental task that required detecting specific stimuli while ignoring other stimuli. We analysed the reliability and the efficacy in the detection of command following resulting from the two methodologies. Results Over repetitive administrations, the BCI paradigm detected command following before the CRS-R in seven patients. Four clinically unresponsive patients consistently showed command following during the BCI assessments. Conclusion Brain-Computer Interface active paradigms might contribute to the evaluation of the level of consciousness, increasing the diagnostic precision of the clinical bedside approach. Significance The integration of different diagnostic methods leads to a better knowledge and care for the DOC.
Collapse
Affiliation(s)
- Rossella Spataro
- IRCCS Centro Neurolesi Bonino Pulejo, Palermo, Italy
- ALS Clinical Research Center, University of Palermo, Palermo, Italy
- *Correspondence: Rossella Spataro,
| | - Yiyan Xu
- ALS Clinical Research Center, University of Palermo, Palermo, Italy
| | - Ren Xu
- g.tec Medical Engineering GmbH, Schiedlberg, Austria
| | - Giorgio Mandalà
- Rehabilitation Unit, Ospedale Buccheri La Ferla, Palermo, Italy
| | - Brendan Z. Allison
- Cognitive Science Department, University of California, San Diego, San Diego, United States
| | - Rupert Ortner
- g.tec Medical Engineering Spain S.L., Barcelona, Spain
| | | | | | - Christoph Guger
- g.tec Medical Engineering GmbH, Schiedlberg, Austria
- g.tec Medical Engineering Spain S.L., Barcelona, Spain
| |
Collapse
|
14
|
Peters B, Eddy B, Galvin-McLaughlin D, Betz G, Oken B, Fried-Oken M. A systematic review of research on augmentative and alternative communication brain-computer interface systems for individuals with disabilities. Front Hum Neurosci 2022; 16:952380. [PMID: 35966988 PMCID: PMC9374067 DOI: 10.3389/fnhum.2022.952380] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2022] [Accepted: 07/04/2022] [Indexed: 11/13/2022] Open
Abstract
Augmentative and alternative communication brain-computer interface (AAC-BCI) systems are intended to offer communication access to people with severe speech and physical impairment (SSPI) without requiring volitional movement. As the field moves toward clinical implementation of AAC-BCI systems, research involving participants with SSPI is essential. Research has demonstrated variability in AAC-BCI system performance across users, and mixed results for comparisons of performance for users with and without disabilities. The aims of this systematic review were to (1) describe study, system, and participant characteristics reported in BCI research, (2) summarize the communication task performance of participants with disabilities using AAC-BCI systems, and (3) explore any differences in performance for participants with and without disabilities. Electronic databases were searched in May, 2018, and March, 2021, identifying 6065 records, of which 73 met inclusion criteria. Non-experimental study designs were common and sample sizes were typically small, with approximately half of studies involving five or fewer participants with disabilities. There was considerable variability in participant characteristics, and in how those characteristics were reported. Over 60% of studies reported an average selection accuracy ≤70% for participants with disabilities in at least one tested condition. However, some studies excluded participants who did not reach a specific system performance criterion, and others did not state whether any participants were excluded based on performance. Twenty-nine studies included participants both with and without disabilities, but few reported statistical analyses comparing performance between the two groups. Results suggest that AAC-BCI systems show promise for supporting communication for people with SSPI, but they remain ineffective for some individuals. The lack of standards in reporting outcome measures makes it difficult to synthesize data across studies. Further research is needed to demonstrate efficacy of AAC-BCI systems for people who experience SSPI of varying etiologies and severity levels, and these individuals should be included in system design and testing. Consensus in terminology and consistent participant, protocol, and performance description will facilitate the exploration of user and system characteristics that positively or negatively affect AAC-BCI use, and support innovations that will make this technology more useful to a broader group of people. Clinical trial registration https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42018095345, PROSPERO: CRD42018095345.
Collapse
Affiliation(s)
- Betts Peters
- Consortium for Accessible Multimodal Brain-Body Interfaces, United States
- REKNEW Projects, Institute on Development and Disability, Department of Pediatrics, Oregon Health and Science University, Portland, OR, United States
| | - Brandon Eddy
- Consortium for Accessible Multimodal Brain-Body Interfaces, United States
- REKNEW Projects, Institute on Development and Disability, Department of Pediatrics, Oregon Health and Science University, Portland, OR, United States
- Speech and Hearing Sciences Department, Portland State University, Portland, OR, United States
| | - Deirdre Galvin-McLaughlin
- Consortium for Accessible Multimodal Brain-Body Interfaces, United States
- REKNEW Projects, Institute on Development and Disability, Department of Pediatrics, Oregon Health and Science University, Portland, OR, United States
| | - Gail Betz
- Health Sciences and Human Services Library, University of Maryland, Baltimore, MD, United States
| | - Barry Oken
- Consortium for Accessible Multimodal Brain-Body Interfaces, United States
- Department of Neurology, Oregon Health & Science University, Portland, OR, United States
| | - Melanie Fried-Oken
- Consortium for Accessible Multimodal Brain-Body Interfaces, United States
- REKNEW Projects, Institute on Development and Disability, Department of Pediatrics, Oregon Health and Science University, Portland, OR, United States
| |
Collapse
|
15
|
Visuo-auditory stimuli with semantic, temporal and spatial congruence for a P300-based BCI: An exploratory test with an ALS patient in a completely locked-in state. J Neurosci Methods 2022; 379:109661. [PMID: 35817307 DOI: 10.1016/j.jneumeth.2022.109661] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2021] [Revised: 05/25/2022] [Accepted: 07/04/2022] [Indexed: 11/23/2022]
Abstract
BACKGROUND Brain-computer interfaces (BCIs) are a promising tool for communication with completely locked-in state (CLIS) patients. Despite the great efforts already made by the BCI research community, the cases of success are still very few, very exploratory, limited in time, and based on simple 'yes/no' paradigms. NEW METHOD A P300-based BCI is proposed comparing two conditions, one corresponding to purely spatial auditory stimuli (AU-S) and the other corresponding to hybrid visual and spatial auditory stimuli (HVA-S). In the HVA-S condition, there is a semantic, temporal, and spatial congruence between visual and auditory stimuli. The stimuli comprise a lexicon of 7 written and spoken words. Spatial sounds are generated through the head-related transfer function. Given the good results obtained with 10 able-bodied participants, we investigated whether a patient entering CLIS could use the proposed BCI. RESULTS The able-bodied group achieved 71.3 % and 90.5 % online classification accuracy for the auditory and hybrid BCIs respectively, while the patient achieved 30 % and chance level accuracies, for the same conditions. Notwithstanding, the patient's event-related potentials (ERPs) showed statistical discrimination between target and non-target events in different time windows. COMPARISON WITH EXISTING METHODS The results of the control group compare favorably with the state-of-the-art, considering a 7-class BCI controlled visual-covertly and with auditory stimuli. The integration of visual and auditory stimuli has not been tested before with CLIS patients. CONCLUSIONS The semantic, temporal, and spatial congruence of the stimuli increased the performance of the control group, but not of the CLIS patient, which can be due to impaired attention and cognitive function. The patient's unique ERP patterns make interpretation difficult, requiring further tests/paradigms to decouple patients' responses at different levels (reflexive, perceptual, cognitive). The ERPs discrimination found indicates that a simplification of the proposed approaches may be feasible.
Collapse
|
16
|
Jadavji Z, Zewdie E, Kelly D, Kinney-Lang E, Robu I, Kirton A. Establishing a Clinical Brain-Computer Interface Program for Children With Severe Neurological Disabilities. Cureus 2022; 14:e26215. [PMID: 35891842 PMCID: PMC9307353 DOI: 10.7759/cureus.26215] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/22/2022] [Indexed: 12/19/2022] Open
Abstract
Background: Children with severe motor impairment but intact cognition are deprived of fundamental human rights. Quadriplegic cerebral palsy is the most common scenario where rehabilitation options remain limited. Brain-computer interfaces (BCI) represent a potential solution, but pediatric populations have been neglected. Direct engagement of children and families could provide meaningful opportunities while informing program development. We describe a patient-centered, clinical, non-invasive pediatric BCI program. Methods: Eligible children were identified within a population-based, tertiary care children’s hospital. Criteria included 1) age six to 18 years, 2) severe physical disability (non-ambulatory, minimal hand use), 3) severely limited speech, and 4) evidence of grade 1 cognitive capacity. After initial screening for BCI competency, participants attended regular sessions, attempting commercially available and customized systems to play computer games, control devices, and attempt communication. Results: We report the first 10 participants (median 11 years, range 6-16, 60% male). Over 334 hours of participation, there were no serious adverse events. BCI training was well tolerated, with favorable feedback from children and parents. All but one participant demonstrated the ability to perform BCI tasks. The majority performed well, using motor imagery based tasks for games and entertainment. Difficulties were most significant using P300, visual evoked potential based paradigms where maintenance of attention was challenging. Children and families expressed interest in continuing and informing program development. Conclusions: Patient-centered clinical BCI programs are feasible for children with severe disabilities. Carefully selected participants can often learn quickly to perform meaningful tasks on readily available systems. Patient and family motivation and engagement appear high.
Collapse
|
17
|
Huang X, Liang S, Li Z, Lai CYY, Choi KS. EEG-based vibrotactile evoked brain-computer interfaces system: A systematic review. PLoS One 2022; 17:e0269001. [PMID: 35657949 PMCID: PMC9165854 DOI: 10.1371/journal.pone.0269001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2021] [Accepted: 05/12/2022] [Indexed: 11/18/2022] Open
Abstract
Recently, a novel electroencephalogram-based brain-computer interface (EVE-BCI) using the vibrotactile stimulus shows great potential for an alternative to other typical motor imagery and visual-based ones. (i) Objective: in this review, crucial aspects of EVE-BCI are extracted from the literature to summarize its key factors, investigate the synthetic evidence of feasibility, and generate recommendations for further studies. (ii) Method: five major databases were searched for relevant publications. Multiple key concepts of EVE-BCI, including data collection, stimulation paradigm, vibrotactile control, EEG signal processing, and reported performance, were derived from each eligible article. We then analyzed these concepts to reach our objective. (iii) Results: (a) seventy-nine studies are eligible for inclusion; (b) EEG data are mostly collected among healthy people with an embodiment of EEG cap in EVE-BCI development; (c) P300 and Steady-State Somatosensory Evoked Potential are the two most popular paradigms; (d) only locations of vibration are heavily explored by previous researchers, while other vibrating factors draw little interest. (e) temporal features of EEG signal are usually extracted and used as the input to linear predictive models for EVE-BCI setup; (f) subject-dependent and offline evaluations remain popular assessments of EVE-BCI performance; (g) accuracies of EVE-BCI are significantly higher than chance levels among different populations. (iv) Significance: we summarize trends and gaps in the current EVE-BCI by identifying influential factors. A comprehensive overview of EVE-BCI can be quickly gained by reading this review. We also provide recommendations for the EVE-BCI design and formulate a checklist for a clear presentation of the research work. They are useful references for researchers to develop a more sophisticated and practical EVE-BCI in future studies.
Collapse
Affiliation(s)
- Xiuyu Huang
- Centre for Smart Health, School of Nursing, The Hong Kong Polytechnic University, Hong Kong SAR, China
- * E-mail:
| | - Shuang Liang
- School of Geographic and Biologic Information, Nanjing University of Posts and Telecommunications, Nanjing, China
| | - Zengguang Li
- School of Computer Science and Technology, Nanjing Tech University, Nanjing, China
| | - Cynthia Yuen Yi Lai
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong SAR, China
| | - Kup-Sze Choi
- Centre for Smart Health, School of Nursing, The Hong Kong Polytechnic University, Hong Kong SAR, China
| |
Collapse
|
18
|
Xiao J, He Y, Yu T, Pan J, Xie Q, Cao C, Zheng H, Huang W, Gu Z, Yu Z, Li Y. Towards Assessment of Sound Localization in Disorders of Consciousness Using a Hybrid Audiovisual Brain-Computer Interface. IEEE Trans Neural Syst Rehabil Eng 2022; 30:1422-1432. [PMID: 35584066 DOI: 10.1109/tnsre.2022.3176354] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Behavioral assessment of sound localization in the Coma Recovery Scale-Revised (CRS-R) poses a significant challenge due to motor disability in patients with disorders of consciousness (DOC). Brain-computer interfaces (BCIs), which can directly detect brain activities related to external stimuli, may thus provide an approach to assess DOC patients without the need for any physical behavior. In this study, a novel audiovisual BCI system was developed to simulate sound localization evaluation in CRS-R. Specifically, there were two alternatively flashed buttons on the left and right sides of the graphical user interface, one of which was randomly chosen as the target. The auditory stimuli of bell sounds were simultaneously presented by the ipsilateral loudspeaker during the flashing of the target button, which prompted patients to selectively attend to the target button. The recorded electroencephalography data were analyzed in real time to detect event-related potentials evoked by the target and further to determine whether the target was attended to or not. A significant BCI accuracy for a patient implied that he/she had sound localization. Among eighteen patients, eleven and four showed sound localization in the BCI and CRS-R, respectively. Furthermore, all patients showing sound localization in the CRS-R were among those detected by our BCI. The other seven patients who had no sound localization behavior in CRS-R were identified by the BCI assessment, and three of them showed improvements in the second CRS-R assessment after the BCI experiment. Thus, the proposed BCI system is promising for assisting the assessment of sound localization and improving the clinical diagnosis of DOC patients.
Collapse
|
19
|
Mascolini A, Niazi IK, Mesin L. Non-linear optimized spatial filter for single-trial identification of movement related cortical potential. Biocybern Biomed Eng 2022. [DOI: 10.1016/j.bbe.2022.02.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
20
|
Huggins JE, Krusienski D, Vansteensel MJ, Valeriani D, Thelen A, Stavisky S, Norton JJS, Nijholt A, Müller-Putz G, Kosmyna N, Korczowski L, Kapeller C, Herff C, Halder S, Guger C, Grosse-Wentrup M, Gaunt R, Dusang AN, Clisson P, Chavarriaga R, Anderson CW, Allison BZ, Aksenova T, Aarnoutse E. Workshops of the Eighth International Brain-Computer Interface Meeting: BCIs: The Next Frontier. BRAIN-COMPUTER INTERFACES 2022; 9:69-101. [PMID: 36908334 PMCID: PMC9997957 DOI: 10.1080/2326263x.2021.2009654] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2021] [Accepted: 11/15/2021] [Indexed: 12/11/2022]
Abstract
The Eighth International Brain-Computer Interface (BCI) Meeting was held June 7-9th, 2021 in a virtual format. The conference continued the BCI Meeting series' interactive nature with 21 workshops covering topics in BCI (also called brain-machine interface) research. As in the past, workshops covered the breadth of topics in BCI. Some workshops provided detailed examinations of specific methods, hardware, or processes. Others focused on specific BCI applications or user groups. Several workshops continued consensus building efforts designed to create BCI standards and increase the ease of comparisons between studies and the potential for meta-analysis and large multi-site clinical trials. Ethical and translational considerations were both the primary topic for some workshops or an important secondary consideration for others. The range of BCI applications continues to expand, with more workshops focusing on approaches that can extend beyond the needs of those with physical impairments. This paper summarizes each workshop, provides background information and references for further study, presents an overview of the discussion topics, and describes the conclusion, challenges, or initiatives that resulted from the interactions and discussion at the workshop.
Collapse
Affiliation(s)
- Jane E Huggins
- Department of Physical Medicine and Rehabilitation, Department of Biomedical Engineering, Neuroscience Graduate Program, University of Michigan, Ann Arbor, Michigan, United States 325 East Eisenhower, Room 3017; Ann Arbor, Michigan 48108-5744, 734-936-7177
| | - Dean Krusienski
- Department of Biomedical Engineering, Virginia Commonwealth University, Richmond, VA 23219
| | - Mariska J Vansteensel
- UMC Utrecht Brain Center, Dept of Neurosurgery, University Medical Center Utrecht, The Netherlands
| | | | - Antonia Thelen
- eemagine Medical Imaging Solutions GmbH, Berlin, Germany
| | | | - James J S Norton
- National Center for Adaptive Neurotechnologies, US Department of Veterans Affairs, 113 Holland Ave, Albany, NY 12208
| | - Anton Nijholt
- Faculty EEMCS, University of Twente, Enschede, The Netherlands
| | - Gernot Müller-Putz
- Institute of Neural Engineering, GrazBCI Lab, Graz University of Technology, Stremayrgasse 16/4, 8010 Graz, Austria
| | - Nataliya Kosmyna
- Massachusetts Institute of Technology (MIT), Media Lab, E14-548, Cambridge, MA 02139, Unites States
| | | | | | - Christian Herff
- School of Mental Health and Neuroscience, Maastricht University, Maastricht, The Netherlands
| | | | - Christoph Guger
- g.tec medical engineering GmbH/Guger Technologies OG, Austria, Sierningstrasse 14, 4521 Schiedlberg, Austria, +43725122240-0
| | - Moritz Grosse-Wentrup
- Research Group Neuroinformatics, Faculty of Computer Science, Vienna Cognitive Science Hub, Data Science @ Uni Vienna University of Vienna
| | - Robert Gaunt
- Rehab Neural Engineering Labs, Department of Physical Medicine and Rehabilitation, Center for the Neural Basis of Cognition, University of Pittsburgh, Pittsburgh, PA, USA, 3520 5th Ave, Suite 300, Pittsburgh, PA 15213, 412-383-1426
| | - Aliceson Nicole Dusang
- Department of Electrical and Computer Engineering, School of Engineering, Brown University, Carney Institute for Brain Science, Brown University, Providence, RI
- Department of Veterans Affairs Medical Center, Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Providence, RI
- Center for Neurotechnology and Neurorecovery, Neurology, Massachusetts General Hospital, Boston, MA
| | | | - Ricardo Chavarriaga
- IEEE Standards Association Industry Connections group on neurotechnologies for brain-machine interface, Center for Artificial Intelligence, School of Engineering, ZHAW-Zurich University of Applied Sciences, Switzerland, Switzerland
| | - Charles W Anderson
- Department of Computer Science, Molecular, Cellular and Integrative Neurosience Program, Colorado State University, Fort Collins, CO 80523
| | - Brendan Z Allison
- Dept. of Cognitive Science, Mail Code 0515, University of California at San Diego, La Jolla, United States, 619-534-9754
| | - Tetiana Aksenova
- University Grenoble Alpes, CEA, LETI, Clinatec, Grenoble 38000, France
| | - Erik Aarnoutse
- UMC Utrecht Brain Center, Department of Neurology & Neurosurgery, University Medical Center Utrecht, Heidelberglaan 100, 3584 CX Utrecht, The Netherlands
| |
Collapse
|
21
|
Xu R, Spataro R, Allison BZ, Guger C. Brain-Computer Interfaces in Acute and Subacute Disorders of Consciousness. J Clin Neurophysiol 2022; 39:32-39. [PMID: 34474428 DOI: 10.1097/wnp.0000000000000810] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022] Open
Abstract
SUMMARY Disorders of consciousness include coma, unresponsive wakefulness syndrome (also known as vegetative state), and minimally conscious state. Neurobehavioral scales such as coma recovery scale-revised are the gold standard for disorder of consciousness assessment. Brain-computer interfaces have been emerging as an alternative tool for these patients. The application of brain-computer interfaces in disorders of consciousness can be divided into four fields: assessment, communication, prediction, and rehabilitation. The operational theoretical model of consciousness that brain-computer interfaces explore was reviewed in this article, with a focus on studies with acute and subacute patients. We then proposed a clinically friendly guideline, which could contribute to the implementation of brain-computer interfaces in neurorehabilitation settings. Finally, we discussed limitations and future directions, including major challenges and possible solutions.
Collapse
Affiliation(s)
- Ren Xu
- Guger Technologies OG, Schiedlberg, Austria
| | - Rossella Spataro
- g.tec medical engineering GmbH, Schiedlberg, Austria
- IRCCS Centro Neurolesi Bonino Pulejo, Palermo, Italy; and
| | - Brendan Z Allison
- Cognitive Science Department, University of California San Diego, La Jolla, California, U.S.A
| | - Christoph Guger
- Guger Technologies OG, Schiedlberg, Austria
- g.tec medical engineering GmbH, Schiedlberg, Austria
| |
Collapse
|
22
|
Nagels-Coune L, Riecke L, Benitez-Andonegui A, Klinkhammer S, Goebel R, De Weerd P, Lührs M, Sorger B. See, Hear, or Feel - to Speak: A Versatile Multiple-Choice Functional Near-Infrared Spectroscopy-Brain-Computer Interface Feasible With Visual, Auditory, or Tactile Instructions. Front Hum Neurosci 2021; 15:784522. [PMID: 34899223 PMCID: PMC8656940 DOI: 10.3389/fnhum.2021.784522] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2021] [Accepted: 11/05/2021] [Indexed: 11/13/2022] Open
Abstract
Severely motor-disabled patients, such as those suffering from the so-called "locked-in" syndrome, cannot communicate naturally. They may benefit from brain-computer interfaces (BCIs) exploiting brain signals for communication and therewith circumventing the muscular system. One BCI technique that has gained attention recently is functional near-infrared spectroscopy (fNIRS). Typically, fNIRS-based BCIs allow for brain-based communication via voluntarily modulation of brain activity through mental task performance guided by visual or auditory instructions. While the development of fNIRS-BCIs has made great progress, the reliability of fNIRS-BCIs across time and environments has rarely been assessed. In the present fNIRS-BCI study, we tested six healthy participants across three consecutive days using a straightforward four-choice fNIRS-BCI communication paradigm that allows answer encoding based on instructions using various sensory modalities. To encode an answer, participants performed a motor imagery task (mental drawing) in one out of four time periods. Answer encoding was guided by either the visual, auditory, or tactile sensory modality. Two participants were tested outside the laboratory in a cafeteria. Answers were decoded from the time course of the most-informative fNIRS channel-by-chromophore combination. Across the three testing days, we obtained mean single- and multi-trial (joint analysis of four consecutive trials) accuracies of 62.5 and 85.19%, respectively. Obtained multi-trial accuracies were 86.11% for visual, 80.56% for auditory, and 88.89% for tactile sensory encoding. The two participants that used the fNIRS-BCI in a cafeteria obtained the best single- (72.22 and 77.78%) and multi-trial accuracies (100 and 94.44%). Communication was reliable over the three recording sessions with multi-trial accuracies of 86.11% on day 1, 86.11% on day 2, and 83.33% on day 3. To gauge the trade-off between number of optodes and decoding accuracy, averaging across two and three promising fNIRS channels was compared to the one-channel approach. Multi-trial accuracy increased from 85.19% (one-channel approach) to 91.67% (two-/three-channel approach). In sum, the presented fNIRS-BCI yielded robust decoding results using three alternative sensory encoding modalities. Further, fNIRS-BCI communication was stable over the course of three consecutive days, even in a natural (social) environment. Therewith, the developed fNIRS-BCI demonstrated high flexibility, reliability and robustness, crucial requirements for future clinical applicability.
Collapse
Affiliation(s)
- Laurien Nagels-Coune
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
- Maastricht Brain Imaging Center, Maastricht, Netherlands
- Zorggroep Sint-Kamillus, Bierbeek, Belgium
| | - Lars Riecke
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
- Maastricht Brain Imaging Center, Maastricht, Netherlands
| | - Amaia Benitez-Andonegui
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
- Maastricht Brain Imaging Center, Maastricht, Netherlands
- MEG Core Facility, National Institutes of Mental Health, Bethesda, MD, United States
| | - Simona Klinkhammer
- Department of Psychiatry and Neuropsychology, Faculty of Health Medicine and Life Sciences, Maastricht University, Maastricht, Netherlands
- School for Mental Health and Neuroscience, Maastricht University, Maastricht, Netherlands
| | - Rainer Goebel
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
- Maastricht Brain Imaging Center, Maastricht, Netherlands
- Brain Innovation B.V., Maastricht, Netherlands
| | - Peter De Weerd
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
- Maastricht Brain Imaging Center, Maastricht, Netherlands
- Maastricht Centre for Systems Biology, Maastricht University, Maastricht, Netherlands
| | | | - Bettina Sorger
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
- Maastricht Brain Imaging Center, Maastricht, Netherlands
| |
Collapse
|
23
|
Adama S, Bogdan M. Yes/No Classification of EEG data from CLIS patients. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:5727-5732. [PMID: 34892421 DOI: 10.1109/embc46164.2021.9629716] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
The goal of this research is to evaluate the usability of new features to classify EEG data from several completely locked-in patients (CLIS), and eventually build a more reliable communication system for them. Patients in such state are completely paralyzed, preventing them to be able to talk, but they retain their cognitive abilities.The data were obtained from four CLIS patients and recorded during an auditory paradigm task during which they were asked yes/no questions. Spectral measures such as the relative power of δ, θ, α, β and γ frequency bands, spectral edge frequencies (SEF50 and SEF95), complexity measure obtained from Poincaré plots and connectivity measures such as the imaginary part of coherency and the weighted Symbolic Mutual Information (wSMI) were used as features. The data was classified using Random Forest and Support Vector Machine, two methods successfully used to classify mental states in both healthy subjects and patients. Additionally, two cases were studied. The first case uses data recorded when the patient is answering questions, while in the second case it also includes data recorded when the experimenter is asking the questions.The classification accuracy during training varies between 51.73 to 67.72% in the first case, and from 50.41 to 67.94% for the second case. Overall, wSMI with a time lag of 64 ms gave the best classification accuracy and in general, Random Forest appears to be the best classification method.Clinical relevance This case study investigates the usability of new features based on EEG complexity and connectivity to classify CLIS patients brain signal, what results in a further step toward the demand of more effective EEG-based Brain-Computer Interface communication systems for CLIS patients.
Collapse
|
24
|
Zapała D, Hossaini A, Kianpour M, Sahonero-Alvarez G, Ayesh A. A functional BCI model by the P2731 working group: psychology. BRAIN-COMPUTER INTERFACES 2021. [DOI: 10.1080/2326263x.2021.1935124] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Affiliation(s)
- Dariusz Zapała
- Department of Experimental Psychology, The John Paul II Catholic University of Lublin, Lublin, Poland
| | - Ali Hossaini
- Department of Engineering, King’s College London, London, UK
| | - Mazaher Kianpour
- Department of Information Security and Communication Technology, Norwegian University of Science and Technology, Gjøvik, Norway
| | - Guillermo Sahonero-Alvarez
- Center for Research, Development and Innovation in Mechatronics Engineering,Department of Mechatronics Engineering, Universidad Catolica Boliviana San Pablo, La Paz, Bolivia
| | - Aladdin Ayesh
- Faculty of Computing,Engineering and Media,De Montfort University, Leicester, UK
| |
Collapse
|
25
|
Sutaj N, Walchshofer M, Schreiner L, Turchet L, Pretl H, Guger C. Evaluating a Novel P300-Based Real-Time Image Ranking BCI. FRONTIERS IN COMPUTER SCIENCE 2021. [DOI: 10.3389/fcomp.2021.661224] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Abstract
Brain–computer interfaces (BCIs) establish communication between a human brain and a computer or external devices by translating the electroencephalography (EEG) signal into computer commands. After stimulating a sensory organ, a positive deflection of the EEG signal between 250 and 700 ms can be measured. This signal component of the event-related potential (ERP) is called “P300.” Numerous studies have provided evidence that the P300 amplitude and latency are linked to sensory perception, engagement, and cognition. Combining the advances in technology, classification methods, and signal processing, we developed a novel image ranking system called the Unicorn Blondy Check. In this study, the application was tested on 21 subjects using three different visual oddball paradigms. Two consisted of female faces and gray-scale images, while the third test paradigm consisted of familiar and unfamiliar faces. The images were displayed for a duration of 150 ms in a randomized order. The system was trained using 50 trials and tested with 30 trials. The EEG data were acquired using the Unicorn Hybrid Black eight-channel BCI system. These synchronized recordings were analyzed, and the achieved classification accuracies were calculated. The EEG signal was averaged over all participants and for every paradigm separately. Analysis of the EEG data revealed a significant shift in the P300 latency dependent on the paradigm and decreased amplitude for a lower target to non-target ratio. The image ranking application achieved a mean accuracy of 100 and 95.5% for ranking female faces above gray-scale images with ratios of 1:11 and 5:11, respectively. In the case of four familiar faces to 24 unfamiliar faces, 86.4% was reached. The obtained results illustrate this novel system’s functionality due to accuracies above chance levels for all subjects.
Collapse
|
26
|
Stieger JR, Engel SA, Suma D, He B. Benefits of deep learning classification of continuous noninvasive brain-computer interface control. J Neural Eng 2021; 18. [PMID: 34038873 DOI: 10.1088/1741-2552/ac0584] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2021] [Accepted: 05/26/2021] [Indexed: 11/12/2022]
Abstract
Objective. Noninvasive brain-computer interfaces (BCIs) assist paralyzed patients by providing access to the world without requiring surgical intervention. Prior work has suggested that EEG motor imagery based BCI can benefit from increased decoding accuracy through the application of deep learning methods, such as convolutional neural networks (CNNs).Approach. Here, we examine whether these improvements can generalize to practical scenarios such as continuous control tasks (as opposed to prior work reporting one classification per trial), whether valuable information remains latent outside of the motor cortex (as no prior work has compared full scalp coverage to motor only electrode montages), and the existing challenges to the practical implementation of deep-learning based continuous BCI control.Main results. We report that: (1) deep learning methods significantly increase offline performance compared to standard methods on an independent, large, and longitudinal online motor imagery BCI dataset with up to 4-classes and continuous 2D feedback; (2) our results suggest that a variety of neural biomarkers for BCI, including those outside the motor cortex, can be detected and used to improve performance through deep learning methods, and (3) tuning neural network output will be an important step in optimizing online BCI control, as we found the CNN models trained with full scalp EEG also significantly reduce the average trial length in a simulated online cursor control environment.Significance. This work demonstrates the benefits of CNNs classification during BCI control while providing evidence that electrode montage selection and the mapping of CNN output to device control will be important design choices in CNN based BCIs.
Collapse
Affiliation(s)
- James R Stieger
- Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213, United States of America.,University of Minnesota, Minneapolis, MN, United States of America
| | - Stephen A Engel
- Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213, United States of America.,University of Minnesota, Minneapolis, MN, United States of America
| | - Daniel Suma
- Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213, United States of America
| | - Bin He
- Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213, United States of America
| |
Collapse
|
27
|
Malhi SK, Welch-West P, Koo AM, Fogarty J, Lazosky A. Thinking without speaking: Neuropsychological testing with individuals who have communication impairments. Neuropsychol Rehabil 2021; 32:1605-1619. [PMID: 33977850 DOI: 10.1080/09602011.2021.1921813] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Abstract
PURPOSE Cognitive ability may be masked by communication impairments. This study aimed to assess cognitive functioning using binary choice (i.e., yes/no) neuropsychological tests in patients with communication impairments. Four participants underwent neuropsychological testing. Two participants were in the minimally conscious state (MCS), one participant had locked-in syndrome and was an alternative communication user, and one participant was an augmentative communication user. There was better performance in all cognitive domains for the augmentative and alternative communication (AAC) users (who performed like the non-communication impaired normative data) compared to the MCS participants. However, using established yes/no communication methods, MCS participants performed above chance on a measure of memory and performance on measures of auditory comprehension was variable. Auditory comprehension appeared to be more influenced by working memory demands for the MCS participants than for the AAC users. For emotional functioning, the AAC users endorsed lower mood compared to the MCS participants. The results support the need to assess cognition, communication, as well as capacity in individuals with communication impairments with the consultation of a neuropsychologist and a speech-language pathologist.
Collapse
Affiliation(s)
- Simritpal Kaur Malhi
- London Health Sciences Centre, London, ON, Canada.,St. Joseph's Health Care London, London, ON, Canada
| | | | | | | | | |
Collapse
|
28
|
Chu C, Luo J, Tian X, Han X, Guo S. A P300 Brain-Computer Interface Paradigm Based on Electric and Vibration Simple Command Tactile Stimulation. Front Hum Neurosci 2021; 15:641357. [PMID: 33935672 PMCID: PMC8081187 DOI: 10.3389/fnhum.2021.641357] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2020] [Accepted: 03/09/2021] [Indexed: 12/15/2022] Open
Abstract
This paper proposed a novel tactile-stimuli P300 paradigm for Brain-Computer Interface (BCI), which potentially targeted at people with less learning ability or difficulty in maintaining attention. The new paradigm using only two types of stimuli was designed, and different targets were distinguished by frequency and spatial information. The classification algorithm was developed by introducing filters for frequency bands selection and conducting optimization with common spatial pattern (CSP) on the tactile evoked EEG signals. It features a combination of spatial and frequency information, with the spatial information distinguishing the sites of stimuli and frequency information identifying target stimuli and disturbances. We investigated both electrical stimuli and vibration stimuli, in which only one target site was stimulated in each block. The results demonstrated an average accuracy of 94.88% for electrical stimuli and 95.21% for vibration stimuli, respectively.
Collapse
Affiliation(s)
- Chenxi Chu
- Institute of Artificial Intelligence (AI) and Robotics, Academy for Engineering and Technology, Fudan University, as well as Engineering Research Center of AI & Robotics, Ministry of Education, Shanghai, China
- Guanghua Lingang Engineering Application and Technology R&D (Shanghai) Co., Ltd., Shanghai, China
| | - Jingjing Luo
- Institute of Artificial Intelligence (AI) and Robotics, Academy for Engineering and Technology, Fudan University, as well as Engineering Research Center of AI & Robotics, Ministry of Education, Shanghai, China
- Jihua Laboratory, Guangzhou, China
| | - Xiwei Tian
- Department of the State Key Laboratory of Reliability and Intelligence of Electrical Equipment and The Hebei Key Laboratory of Robot Perception and Human-Robot Interaction, Hebei University of Technology, Tianjin, China
| | - Xiangke Han
- Department of the State Key Laboratory of Reliability and Intelligence of Electrical Equipment and The Hebei Key Laboratory of Robot Perception and Human-Robot Interaction, Hebei University of Technology, Tianjin, China
| | - Shijie Guo
- Institute of Artificial Intelligence (AI) and Robotics, Academy for Engineering and Technology, Fudan University, as well as Engineering Research Center of AI & Robotics, Ministry of Education, Shanghai, China
- Guanghua Lingang Engineering Application and Technology R&D (Shanghai) Co., Ltd., Shanghai, China
- Department of the State Key Laboratory of Reliability and Intelligence of Electrical Equipment and The Hebei Key Laboratory of Robot Perception and Human-Robot Interaction, Hebei University of Technology, Tianjin, China
| |
Collapse
|
29
|
Stieger JR, Engel SA, He B. Continuous sensorimotor rhythm based brain computer interface learning in a large population. Sci Data 2021; 8:98. [PMID: 33795705 PMCID: PMC8016873 DOI: 10.1038/s41597-021-00883-1] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2020] [Accepted: 02/19/2021] [Indexed: 02/01/2023] Open
Abstract
Brain computer interfaces (BCIs) are valuable tools that expand the nature of communication through bypassing traditional neuromuscular pathways. The non-invasive, intuitive, and continuous nature of sensorimotor rhythm (SMR) based BCIs enables individuals to control computers, robotic arms, wheel-chairs, and even drones by decoding motor imagination from electroencephalography (EEG). Large and uniform datasets are needed to design, evaluate, and improve the BCI algorithms. In this work, we release a large and longitudinal dataset collected during a study that examined how individuals learn to control SMR-BCIs. The dataset contains over 600 hours of EEG recordings collected during online and continuous BCI control from 62 healthy adults, (mostly) right hand dominant participants, across (up to) 11 training sessions per participant. The data record consists of 598 recording sessions, and over 250,000 trials of 4 different motor-imagery-based BCI tasks. The current dataset presents one of the largest and most complex SMR-BCI datasets publicly available to date and should be useful for the development of improved algorithms for BCI control.
Collapse
Affiliation(s)
- James R Stieger
- Carnegie Mellon University, Pittsburgh, PA, USA
- University of Minnesota, Minneapolis, MN, USA
| | | | - Bin He
- Carnegie Mellon University, Pittsburgh, PA, USA.
| |
Collapse
|
30
|
Fernández-Rodríguez Á, Medina-Juliá MT, Velasco-Álvarez F, Ron-Angevin R. Different effects of using pictures as stimuli in a P300 brain-computer interface under rapid serial visual presentation or row-column paradigm. Med Biol Eng Comput 2021; 59:869-881. [PMID: 33742353 DOI: 10.1007/s11517-021-02340-y] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2020] [Accepted: 02/23/2021] [Indexed: 02/08/2023]
Abstract
Previous proposals for controlling a P300-based BCI speller have shown an improvement using alternative images instead of letters as target stimuli under a row-column paradigm (RCP). However, the RCP is not suitable for those patients with a lack of gaze control. To solve that, the rapid serial visual presentation (RSVP) paradigm has been proposed in previous studies. The aim of the present work is to assess if a set of alternative pictures that improved performance in RCP could also improve performance in RSVP. Sixteen participants controlled four conditions in calibration and online tasks: letters in RCP, pictures in RCP, letters in RSVP and pictures in RSVP. The effect given by pictures was greater under RCP than under RSVP, both for performance and event-related potential analyses. Indeed, pictures did not show any improvement under RSVP in comparison to letters. In addition, the condition with pictures under RCP was declared the favourite by most users (68.75%), while the condition with pictures under RSVP was not chosen as favourite by any participant. Therefore, this work shows that the improvement related to the use of pictures as alternative flashing stimuli under RCP may not be transferred to RSVP. Graphical abstract.
Collapse
Affiliation(s)
- Álvaro Fernández-Rodríguez
- Departamento de Tecnología Electrónica, Universidad de Málaga, 35 Louis Pasteur Boulevard, 29071, Malaga, Spain.
| | - María Teresa Medina-Juliá
- Departamento de Tecnología Electrónica, Universidad de Málaga, 35 Louis Pasteur Boulevard, 29071, Malaga, Spain
| | - Francisco Velasco-Álvarez
- Departamento de Tecnología Electrónica, Universidad de Málaga, 35 Louis Pasteur Boulevard, 29071, Malaga, Spain
| | - Ricardo Ron-Angevin
- Departamento de Tecnología Electrónica, Universidad de Málaga, 35 Louis Pasteur Boulevard, 29071, Malaga, Spain
| |
Collapse
|
31
|
Georgiadis K, Adamos DA, Nikolopoulos S, Laskaris N, Kompatsiaris I. Covariation Informed Graph Slepians for Motor Imagery Decoding. IEEE Trans Neural Syst Rehabil Eng 2021; 29:340-349. [PMID: 33417560 DOI: 10.1109/tnsre.2021.3049998] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Graph signal processing (GSP) provides signal analytic tools for data defined in irregular domains, as is the case of non-invasive electroencephalography (EEG). In this work, the recently introduced technique of Graph Slepian functions is exploited for the robust decoding of motor imagery (MI) brain activity. The particular technique builds over the concept of graph Fourier transform (GFT) and provides additional flexibility in the subsequent data analysis by incorporating domain knowledge. Based on contrastive learning, we introduce an algorithmic pipeline that attains a data driven and subject specific design of Graph Slepian functions. These functions, by incorporating both the topology of the sensor array and the empirical evidence about the differential functional covariation, act as spatial filters that enhance the information conveyed by the multichannel signal and specifically relates to the participant's intention. The proposed technique for crafting Graph Slepians is incorporated in a MI-decoding scheme, in which the informed projections are fed to a support vector machine (SVM) that casts a prediction regarding the type of intended movement. The employed MI-decoder is evaluated based on two publicly available datasets and its superiority against popular alternatives in the field is established. Computational efficiency is listed among its main advantages, since it involves only simple matrix operations, allowing to consider its use in real-time implementations.
Collapse
|
32
|
Reichert C, Tellez Ceja IF, Sweeney-Reed CM, Heinze HJ, Hinrichs H, Dürschmid S. Impact of Stimulus Features on the Performance of a Gaze-Independent Brain-Computer Interface Based on Covert Spatial Attention Shifts. Front Neurosci 2020; 14:591777. [PMID: 33335470 PMCID: PMC7736242 DOI: 10.3389/fnins.2020.591777] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2020] [Accepted: 11/10/2020] [Indexed: 11/18/2022] Open
Abstract
Regaining communication abilities in patients who are unable to speak or move is one of the main goals in decoding brain waves for brain-computer interface (BCI) control. Many BCI approaches designed for communication rely on attention to visual stimuli, commonly applying an oddball paradigm, and require both eye movements and adequate visual acuity. These abilities may, however, be absent in patients who depend on BCI communication. We have therefore developed a response-based communication BCI, which is independent of gaze shifts but utilizes covert shifts of attention to the left or right visual field. We recorded the electroencephalogram (EEG) from 29 channels and coregistered the vertical and horizontal electrooculogram. Data-driven decoding of small attention-based differences between the hemispheres, also known as N2pc, was performed using 14 posterior channels, which are expected to reflect correlates of visual spatial attention. Eighteen healthy participants responded to 120 statements by covertly directing attention to one of two colored symbols (green and red crosses for "yes" and "no," respectively), presented in the user's left and right visual field, respectively, while maintaining central gaze fixation. On average across participants, 88.5% (std: 7.8%) of responses were correctly decoded online. In order to investigate the potential influence of stimulus features on accuracy, we presented the symbols with different visual angles, by altering symbol size and eccentricity. The offline analysis revealed that stimulus features have a minimal impact on the controllability of the BCI. Hence, we show with our novel approach that spatial attention to a colored symbol is a robust method with which to control a BCI, which has the potential to support severely paralyzed people with impaired eye movements and low visual acuity in communicating with their environment.
Collapse
Affiliation(s)
- Christoph Reichert
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Magdeburg, Germany
- Center for Behavioral Brain Sciences, Magdeburg, Germany
- Research Campus STIMULATE, Magdeburg, Germany
| | | | - Catherine M. Sweeney-Reed
- Center for Behavioral Brain Sciences, Magdeburg, Germany
- Department of Neurology, Otto-von-Guericke University, Magdeburg, Germany
| | - Hans-Jochen Heinze
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Magdeburg, Germany
- Center for Behavioral Brain Sciences, Magdeburg, Germany
- Department of Neurology, Otto-von-Guericke University, Magdeburg, Germany
| | - Hermann Hinrichs
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Magdeburg, Germany
- Center for Behavioral Brain Sciences, Magdeburg, Germany
- Research Campus STIMULATE, Magdeburg, Germany
- Department of Neurology, Otto-von-Guericke University, Magdeburg, Germany
| | - Stefan Dürschmid
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Magdeburg, Germany
- Department of Neurology, Otto-von-Guericke University, Magdeburg, Germany
| |
Collapse
|
33
|
Wang L, Zhang Z, Han D, Zhang Z, Liu Z, Liu W. Single stimulus location for two inputs: A combined brain-computer interface based on Steady-State Visual Evoked Potential (SSVEP). Eur J Neurosci 2020; 53:861-875. [PMID: 33128787 DOI: 10.1111/ejn.15030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2020] [Revised: 10/20/2020] [Accepted: 10/21/2020] [Indexed: 11/26/2022]
Abstract
Brain-computer interfaces (BCI) help severely paralyzed people communicate with the outside world. One type of BCI depends on eye movements and has high information transfer (ITR) but is tiring for users and not applicable to people with eye dyskinesia. Conversely, independent BCIs enable attention shifts across visual stimuli without eye movement, but at the cost of a lower ITR. Steady-state visual evoked potential (SSVEP) is an oscillatory brain response and typically used as BCI signal sources because of high signal-to-noise ratio (SNR). Considering the effect of attentional modulation on the SSVEP, we proposed the novel concept of one-to-two BCI to optimize existing problems, wherein the target and other stimuli shared the same location. Specifically, two spatially overlapping stimuli were displayed in the center-of-view field, as in the independent BCI, and participants were required to divide their attention between the right and left visual fields, as in the dependent BCI. Using three different design schemes in two experiments, we aimed to provide a new framework for BCI design by exploring the feasibility of a combined BCI that can realize a single stimulus location for two inputs. The results strongly demonstrated that, even when the targets and distractors overlapped spatially, the former evoked stronger SSVEP responses. Notably, the BCI scheme based on the object-based attention could achieve a recognition rate as high as 83.2% and an ITR of 12.5 bits per minute. The feasibility of a one-to-two BCI design, which simplified the keyboard layout, reduced the attention shift, and relieved user fatigue, was established.
Collapse
Affiliation(s)
- Lu Wang
- Department of Psychology and Behavioral Sciences, Zhejiang University, Hangzhou, China
| | - Zhenhao Zhang
- Department of Psychology and Behavioral Sciences, Zhejiang University, Hangzhou, China
| | - Dan Han
- Department of Psychology and Behavioral Sciences, Zhejiang University, Hangzhou, China
| | - Zhijun Zhang
- Department of Psychology and Behavioral Sciences, Zhejiang University, Hangzhou, China
| | - Zhifang Liu
- Department of Psychology and Special Education, Hangzhou Normal University, Hangzhou, China
| | - Wei Liu
- Department of Education, Dali University, Dali, China
| |
Collapse
|
34
|
Annen J, Mertel I, Xu R, Chatelle C, Lesenfants D, Ortner R, Bonin EA, Guger C, Laureys S, Müller F. Auditory and Somatosensory P3 Are Complementary for the Assessment of Patients with Disorders of Consciousness. Brain Sci 2020; 10:E748. [PMID: 33080842 PMCID: PMC7602953 DOI: 10.3390/brainsci10100748] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2020] [Revised: 09/30/2020] [Accepted: 10/14/2020] [Indexed: 11/24/2022] Open
Abstract
The evaluation of the level of consciousness in patients with disorders of consciousness (DOC) is primarily based on behavioural assessments. Patients with unresponsive wakefulness syndrome (UWS) do not show any sign of awareness of their environment, while minimally conscious state (MCS) patients show reproducible but fluctuating signs of awareness. Some patients, although with remaining cognitive abilities, are not able to exhibit overt voluntary responses at the bedside and may be misdiagnosed as UWS. Several studies investigated functional neuroimaging and neurophysiology as an additional tool to evaluate the level of consciousness and to detect covert command following in DOC. Most of these studies are based on auditory stimulation, neglecting patients suffering from decreased or absent hearing abilities. In the present study, we aim to assess the response to a P3-based paradigm in 40 patients with DOC and 12 healthy participants using auditory (AEP) and vibrotactile (VTP) stimulation. To this end, an EEG-based brain-computer interface was used at DOC patient's bedside. We compared the significance of the P3 performance (i.e., the interpretation of significance of the evoked P3 response) as obtained by 'direct processing' (i.e., theoretical-based significance threshold) and 'offline processing' (i.e., permutation-based single subject level threshold). We evaluated whether the P3 performances were dependent on clinical variables such as diagnosis (UWS and MCS), aetiology and time since injury. Last we tested the dependency of AEP and VTP performances at the single subject level. Direct processing tends to overestimate P3 performance. We did not find any difference in the presence of a P3 performance according to the level of consciousness (UWS vs. MCS) or the aetiology (traumatic vs. non-traumatic brain injury). The performance achieved at the AEP paradigm was independent from what was achieved at the VTP paradigm, indicating that some patients performed better on the AEP task while others performed better on the VTP task. Our results support the importance of using multimodal approaches in the assessment of DOC patients in order to optimise the evaluation of patient's abilities.
Collapse
Affiliation(s)
- Jitka Annen
- GIGA Consciousness, Coma Science Group, University of Liege, 4000 Liege, Belgium; (C.C.); (E.A.C.B.); (S.L.)
- Centre du Cerveau (C2), University Hospital Liege, 4000 Liege, Belgium
| | - Isabella Mertel
- Schoen Klinik Bad Aibling, 83043 Bad Aibling, Germany; (I.M.); (F.M.)
- Department of Clinical Psychology, University of Tuebingen-, 72074 Tuebingen, Germany
| | - Ren Xu
- Guger Technologies OG, 8020 Graz, Austria; (R.X.); (C.G.)
| | - Camille Chatelle
- GIGA Consciousness, Coma Science Group, University of Liege, 4000 Liege, Belgium; (C.C.); (E.A.C.B.); (S.L.)
- Centre du Cerveau (C2), University Hospital Liege, 4000 Liege, Belgium
- Laboratory for NeuroImaging of Coma and Consciousness—Department of Neurology, Massachusetts General Hospital, Harvard Medical School, Boston, 02114 MA, USA
| | - Damien Lesenfants
- Experimental Oto-rino-laryngology, Department of Neuroscience, Katholieke Universiteit Leuven, 3000 Leuven, Belgium;
| | | | - Estelle A.C. Bonin
- GIGA Consciousness, Coma Science Group, University of Liege, 4000 Liege, Belgium; (C.C.); (E.A.C.B.); (S.L.)
- Centre du Cerveau (C2), University Hospital Liege, 4000 Liege, Belgium
- Experimental Oto-rino-laryngology, Department of Neuroscience, Katholieke Universiteit Leuven, 3000 Leuven, Belgium;
| | - Christoph Guger
- Guger Technologies OG, 8020 Graz, Austria; (R.X.); (C.G.)
- g.tec Medical Engineering GmbH, 4521 Schiedlberg, Austria
| | - Steven Laureys
- GIGA Consciousness, Coma Science Group, University of Liege, 4000 Liege, Belgium; (C.C.); (E.A.C.B.); (S.L.)
- Centre du Cerveau (C2), University Hospital Liege, 4000 Liege, Belgium
| | - Friedemann Müller
- Schoen Klinik Bad Aibling, 83043 Bad Aibling, Germany; (I.M.); (F.M.)
| |
Collapse
|
35
|
Fleury M, Lioi G, Barillot C, Lécuyer A. A Survey on the Use of Haptic Feedback for Brain-Computer Interfaces and Neurofeedback. Front Neurosci 2020; 14:528. [PMID: 32655347 PMCID: PMC7325479 DOI: 10.3389/fnins.2020.00528] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2020] [Accepted: 04/28/2020] [Indexed: 11/23/2022] Open
Abstract
Neurofeedback (NF) and brain-computer interface (BCI) applications rely on the registration and real-time feedback of individual patterns of brain activity with the aim of achieving self-regulation of specific neural substrates or control of external devices. These approaches have historically employed visual stimuli. However, in some cases vision is unsuitable or inadequately engaging. Other sensory modalities, such as auditory or haptic feedback have been explored, and multisensory stimulation is expected to improve the quality of the interaction loop. Moreover, for motor imagery tasks, closing the sensorimotor loop through haptic feedback may be relevant for motor rehabilitation applications, as it can promote plasticity mechanisms. This survey reviews the various haptic technologies and describes their application to BCIs and NF. We identify major trends in the use of haptic interfaces for BCI and NF systems and discuss crucial aspects that could motivate further studies.
Collapse
Affiliation(s)
- Mathis Fleury
- University of Rennes 1, INRIA, EMPENN & HYBRID, Rennes, France
| | | | | | | |
Collapse
|
36
|
A Tactile-based Brain Computer Interface P300 Paradigm Using Vibration Frequency and Spatial Location. J Med Biol Eng 2020. [DOI: 10.1007/s40846-020-00535-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
|
37
|
Murovec N, Heilinger A, Xu R, Ortner R, Spataro R, La Bella V, Miao Y, Jin J, Chatelle C, Laureys S, Allison BZ, Guger C. Effects of a Vibro-Tactile P300 Based Brain-Computer Interface on the Coma Recovery Scale-Revised in Patients With Disorders of Consciousness. Front Neurosci 2020; 14:294. [PMID: 32327970 PMCID: PMC7161577 DOI: 10.3389/fnins.2020.00294] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2019] [Accepted: 03/13/2020] [Indexed: 11/22/2022] Open
Abstract
Persons diagnosed with disorders of consciousness (DOC) typically suffer from motor and cognitive disabilities. Recent research has shown that non-invasive brain-computer interface (BCI) technology could help assess these patients' cognitive functions and command following abilities. 20 DOC patients participated in the study and performed 10 vibro-tactile P300 BCI sessions over 10 days with 8-12 runs each day. Vibrotactile tactors were placed on the each patient's left and right wrists and one foot. Patients were instructed, via earbuds, to concentrate and silently count vibrotactile pulses on either their left or right wrist that presented a target stimulus and to ignore the others. Changes of the BCI classification accuracy were investigated over the 10 days. In addition, the Coma Recovery Scale-Revised (CRS-R) score was measured before and after the 10 vibro-tactile P300 sessions. In the first run, 10 patients had a classification accuracy above chance level (>12.5%). In the best run, every patient reached an accuracy ≥60%. The grand average accuracy in the first session for all patients was 40%. In the best session, the grand average accuracy was 88% and the median accuracy across all sessions was 21%. The CRS-R scores compared before and after 10 VT3 sessions for all 20 patients, are showing significant improvement (p = 0.024). Twelve of the twenty patients showed an improvement of 1 to 7 points in the CRS-R score after the VT3 BCI sessions (mean: 2.6). Six patients did not show a change of the CRS-R and two patients showed a decline in the score by 1 point. Every patient achieved at least 60% accuracy at least once, which indicates successful command following. This shows the importance of repeated measures when DOC patients are assessed. The improvement of the CRS-R score after the 10 VT3 sessions is an important issue for future experiments to test the possible therapeutic applications of vibro-tactile and related BCIs with a larger patient group.
Collapse
Affiliation(s)
- Nensi Murovec
- g. tec Medical Engineering GmbH, Schiedlberg, Austria
- Guger Technologies OG, Graz, Austria
| | | | - Ren Xu
- Guger Technologies OG, Graz, Austria
| | - Rupert Ortner
- g. tec Medical Engineering Spain S.L., Barcelona, Spain
| | - Rossella Spataro
- g. tec Medical Engineering GmbH, Schiedlberg, Austria
- IRCCS Centro Neurolesi Bonino Pulejo, Palermo, Italy
| | - Vincenzo La Bella
- ALS Clinical Research Center, Bi.N.D., University of Palermo, Palermo, Italy
| | - Yangyang Miao
- Department of Automation, East China University of Science and Technology, Shanghai, China
| | - Jing Jin
- Department of Automation, East China University of Science and Technology, Shanghai, China
| | - Camille Chatelle
- GIGA Consciousness, Coma Science Group, University of Liège, Liège, Belgium
| | - Steven Laureys
- GIGA Consciousness, Coma Science Group, University of Liège, Liège, Belgium
- French Association of Locked-in Syndrome (ALIS), Paris, France
| | - Brendan Z. Allison
- Department of Cognitive Science, University of California, San Diego, La Jolla, CA, United States
| | - Christoph Guger
- g. tec Medical Engineering GmbH, Schiedlberg, Austria
- Guger Technologies OG, Graz, Austria
| |
Collapse
|
38
|
Fernández-Rodríguez Á, Medina-Juliá MT, Velasco-Álvarez F, Ron-Angevin R. Effects of Spatial Stimulus Overlap in a Visual P300-based Brain-computer Interface. Neuroscience 2020; 431:134-142. [PMID: 32081721 DOI: 10.1016/j.neuroscience.2020.02.011] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2019] [Revised: 02/07/2020] [Accepted: 02/10/2020] [Indexed: 11/19/2022]
Abstract
The rapid serial visual presentation (RSVP) paradigm seems to be one of the most appropriate for patients using P300-based brain-computer interface (BCI) applications, since non-ocular movements are required. However, according to previous works, the use of different locations for each stimulus may improve performance. Thus, the aim of the present work is to explore how spatial overlap between stimuli influences performance in controlling a visual P300-based BCI. Nineteen participants were tested using four levels of overlap between two stimuli: 100%, 66.7%, 33.3% and 0%. Significant differences in accuracy were found between the 0% overlapped condition and all the other conditions, and between 33.3% and higher overlap (66.7% and 100%). These results can be explained due to a modulation in the non-target stimulus amplitude signal caused by the overlapping factor. In short, the stimulus overlap provokes a modulation in performance using a P300-based BCI; this should be considered in future BCI proposals in which an optimal surface exploitation is convenient and potential users have only residual ocular movement.
Collapse
Affiliation(s)
| | | | | | - Ricardo Ron-Angevin
- Departamento de Tecnología Electrónica, Universidad de Málaga, 29071 Malaga, Spain
| |
Collapse
|
39
|
Borgheai SB, McLinden J, Zisk AH, Hosni SI, Deligani RJ, Abtahi M, Mankodiya K, Shahriari Y. Enhancing Communication for People in Late-Stage ALS Using an fNIRS-Based BCI System. IEEE Trans Neural Syst Rehabil Eng 2020; 28:1198-1207. [PMID: 32175867 DOI: 10.1109/tnsre.2020.2980772] [Citation(s) in RCA: 27] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
OBJECTIVE Brain-computer interface (BCI) based communication remains a challenge for people with later-stage amyotrophic lateral sclerosis (ALS) who lose all voluntary muscle control. Although recent studies have demonstrated the feasibility of functional near-infrared spectroscopy (fNIRS) to successfully control BCIs primarily for healthy cohorts, these systems are yet inefficient for people with severe motor disabilities like ALS. In this study, we developed a new fNIRS-based BCI system in concert with a single-trial Visuo-Mental (VM) paradigm to investigate the feasibility of enhanced communication for ALS patients, particularly those in the later stages of the disease. METHODS In the first part of the study, we recorded data from six ALS patients using our proposed protocol (fNIRS-VM) and compared the results with the conventional electroencephalography (EEG)-based multi-trial P3Speller (P3S). In the second part, we recorded longitudinal data from one patient in the late locked-in state (LIS) who had fully lost eye-gaze control. Using statistical parametric mapping (SPM) and correlation analysis, the optimal channels and hemodynamic features were selected and used in linear discriminant analysis (LDA). RESULTS Over all the subjects, we obtained an average accuracy of 81.3%±5.7% within comparatively short times (< 4 sec) in the fNIRS-VM protocol relative to an average accuracy of 74.0%±8.9% in the P3S, though not competitive in patients with no substantial visual problems. Our longitudinal analysis showed substantially superior accuracy using the proposed fNIRS-VM protocol (73.2%±2.0%) over the P3S (61.8%±1.5%). SIGNIFICANCE Our findings indicate the potential efficacy of our proposed system for communication and control for late-stage ALS patients.
Collapse
|
40
|
Sosnik R, Ben Zur O. Reconstruction of hand, elbow and shoulder actual and imagined trajectories in 3D space using EEG slow cortical potentials. J Neural Eng 2020; 17:016065. [DOI: 10.1088/1741-2552/ab59a7] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
|
41
|
Annen J, Laureys S, Gosseries O. Brain-computer interfaces for consciousness assessment and communication in severely brain-injured patients. BRAIN-COMPUTER INTERFACES 2020; 168:137-152. [DOI: 10.1016/b978-0-444-63934-9.00011-1] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
|
42
|
Abstract
Locked-in syndrome (LIS) is characterized by an inability to move or speak in the presence of intact cognition and can be caused by brainstem trauma or neuromuscular disease. Quality of life (QoL) in LIS is strongly impaired by the inability to communicate, which cannot always be remedied by traditional augmentative and alternative communication (AAC) solutions if residual muscle activity is insufficient to control the AAC device. Brain-computer interfaces (BCIs) may offer a solution by employing the person's neural signals instead of relying on muscle activity. Here, we review the latest communication BCI research using noninvasive signal acquisition approaches (electroencephalography, functional magnetic resonance imaging, functional near-infrared spectroscopy) and subdural and intracortical implanted electrodes, and we discuss current efforts to translate research knowledge into usable BCI-enabled communication solutions that aim to improve the QoL of individuals with LIS.
Collapse
|
43
|
Huggins JE, Guger C, Aarnoutse E, Allison B, Anderson CW, Bedrick S, Besio W, Chavarriaga R, Collinger JL, Do AH, Herff C, Hohmann M, Kinsella M, Lee K, Lotte F, Müller-Putz G, Nijholt A, Pels E, Peters B, Putze F, Rupp R, Schalk G, Scott S, Tangermann M, Tubig P, Zander T. Workshops of the Seventh International Brain-Computer Interface Meeting: Not Getting Lost in Translation. BRAIN-COMPUTER INTERFACES 2019; 6:71-101. [PMID: 33033729 PMCID: PMC7539697 DOI: 10.1080/2326263x.2019.1697163] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2019] [Accepted: 10/30/2019] [Indexed: 12/11/2022]
Abstract
The Seventh International Brain-Computer Interface (BCI) Meeting was held May 21-25th, 2018 at the Asilomar Conference Grounds, Pacific Grove, California, United States. The interactive nature of this conference was embodied by 25 workshops covering topics in BCI (also called brain-machine interface) research. Workshops covered foundational topics such as hardware development and signal analysis algorithms, new and imaginative topics such as BCI for virtual reality and multi-brain BCIs, and translational topics such as clinical applications and ethical assumptions of BCI development. BCI research is expanding in the diversity of applications and populations for whom those applications are being developed. BCI applications are moving toward clinical readiness as researchers struggle with the practical considerations to make sure that BCI translational efforts will be successful. This paper summarizes each workshop, providing an overview of the topic of discussion, references for additional information, and identifying future issues for research and development that resulted from the interactions and discussion at the workshop.
Collapse
Affiliation(s)
- Jane E Huggins
- Department of Physical Medicine and Rehabilitation, Department of Biomedical Engineering, Neuroscience Graduate Program, University of Michigan, Ann Arbor, Michigan, United States, 325 East Eisenhower, Room 3017; Ann Arbor, Michigan 48108-5744
| | - Christoph Guger
- g.tec medical engineering GmbH/Guger Technologies OG, Austria, Sierningstrasse 14, 4521 Schiedlberg, Austria
| | - Erik Aarnoutse
- UMC Utrecht Brain Center, Department of Neurology & Neurosurgery, University Medical Center Utrecht, Heidelberglaan 100, 3584 CX Utrecht, The Netherlands
| | - Brendan Allison
- Dept. of Cognitive Science, Mail Code 0515, University of California at San Diego, La Jolla, United States
| | - Charles W Anderson
- Department of Computer Science, Molecular, Cellular and Integrative Neurosience Program, Colorado State University, Fort Collins, CO 80523
| | - Steven Bedrick
- Center for Spoken Language Understanding, Oregon Health & Science University, Portland, OR 97239
| | - Walter Besio
- Department of Electrical, Computer, & Biomedical Engineering and Interdisciplinary Neuroscience Program, University of Rhode Island, Kingston, Rhode Island, USA, CREmedical Corp. Kingston, Rhode Island, USA
| | - Ricardo Chavarriaga
- Defitech Chair in Brain-Machine Interface (CNBI), Center for Neuroprosthetics, Ecole Polytechnique Fédérale de Lausanne - EPFL, Switzerland
| | - Jennifer L Collinger
- University of Pittsburgh, Department of Physical Medicine and Rehabilitation, VA Pittsburgh Healthcare System, Department of Veterans Affairs, 3520 5th Ave, Pittsburgh, PA, 15213
| | - An H Do
- UC Irvine Brain Computer Interface Lab, Department of Neurology, University of California, Irvine
| | - Christian Herff
- School of Mental Health and Neuroscience, Maastricht University, Maastricht, The Netherlands
| | - Matthias Hohmann
- Max Planck Institute for Intelligent Systems, Department for Empirical Inference, Max-Planck-Ring 4, 72074 Tübingen, Germany
| | - Michelle Kinsella
- Oregon Health & Science University, Institute on Development & Disability, 707 SW Gaines St, #1290, Portland, OR 97239
| | - Kyuhwa Lee
- Swiss Federal Institute of Technology in Lausanne-EPFL
| | - Fabien Lotte
- Inria Bordeaux Sud-Ouest, LaBRI (Univ. Bordeaux/CNRS/Bordeaux INP), 200 avenue de la vieille tour, 33405, Talence Cedex, France
| | | | - Anton Nijholt
- Faculty EEMCS, University of Twente, Enschede, The Netherlands
| | - Elmar Pels
- UMC Utrecht Brain Center, Department of Neurology & Neurosurgery, University Medical Center Utrecht, Heidelberglaan 100, 3584 CX Utrecht, The Netherlands
| | - Betts Peters
- Oregon Health & Science University, Institute on Development & Disability, 707 SW Gaines St, #1290, Portland, OR 97239
| | - Felix Putze
- University of Bremen, Germany, Cognitive Systems Lab, University of Bremen, Enrique-Schmidt-Straße 5 (Cartesium), 28359 Bremen
| | - Rüdiger Rupp
- Spinal Cord Injury Center, Heidelberg University Hospital
| | - Gerwin Schalk
- National Center for Adaptive Neurotechnologies, Wadsworth Center, NYS Dept. of Health, Dept. of Neurology, Albany Medical College, Dept. of Biomed. Sci., State Univ. of New York at Albany, Center for Medical Sciences 2003, 150 New Scotland Avenue, Albany, New York 12208
| | - Stephanie Scott
- Department of Media Communications, Colorado State University, Fort Collins, CO 80523
| | - Michael Tangermann
- Brain State Decoding Lab, Cluster of Excellence BrainLinks-BrainTools, Computer Science Dept., University of Freiburg, Germany, Autonomous Intelligent Systems Lab, Computer Science Dept., University of Freiburg, Germany
| | - Paul Tubig
- Department of Philosophy, Center for Neurotechnology, University of Washington, Savery Hall, Room 361, Seattle, WA 98195
| | - Thorsten Zander
- Team PhyPA, Biological Psychology and Neuroergonomics, Technische Universität Berlin, Berlin, Germany, 7 Zander Laboratories B.V., Amsterdam, The Netherlands
| |
Collapse
|
44
|
de Neeling M, Van Hulle MM. Single-paradigm and hybrid brain computing interfaces and their use by disabled patients. J Neural Eng 2019; 16:061001. [DOI: 10.1088/1741-2552/ab2706] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
|
45
|
Shahriari Y, Vaughan TM, McCane LM, Allison BZ, Wolpaw JR, Krusienski DJ. An exploration of BCI performance variations in people with amyotrophic lateral sclerosis using longitudinal EEG data. J Neural Eng 2019; 16:056031. [PMID: 31108477 DOI: 10.1088/1741-2552/ab22ea] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
OBJECTIVE Brain-computer interface (BCI) technology enables people to use direct measures of brain activity for communication and control. The National Center for Adaptive Neurotechnologies and Helen Hayes Hospital are studying long-term independent home use of P300-based BCIs by people with amyotrophic lateral sclerosis (ALS). This BCI use takes place without technical oversight, and users can encounter substantial variation in their day-to-day BCI performance. The purpose of this study is to identify and evaluate features in the electroencephalogram (EEG) that correlate with successful BCI performance during home use with the goal of improving BCI for people with neuromuscular disorders. APPROACH Nine people with ALS used a P300-based BCI at home over several months for communication and computer control. Sessions from a routine calibration task were categorized as successful ([Formula: see text]70%) or unsuccessful (<70%) BCI performance. The correlation of temporal and spectral EEG features with BCI performance was then evaluated. MAIN RESULTS BCI performance was positively correlated with an increase in alpha-band (8-14 Hz) activity at locations PO8, P3, Pz, and P4; and beta-band (15-30 Hz) activity at occipital locations. In addition, performance was significantly positively correlated with a positive deflection in EEG amplitude around 220 ms at frontal mid-line locations (i.e. Fz and Cz). BCI performance was negatively correlated with delta-band (1-3 Hz) activity recorded from occipital locations. SIGNIFICANCE These results highlight the variability found in the EEG and describe EEG features that correlate with successful BCI performance during day-to-day use of a P300-based BCI by people with ALS. These results should inform studies focused on improved BCI reliability for people with neuromuscular disorders.
Collapse
Affiliation(s)
- Y Shahriari
- Biomedical Engineering, University of Rhode Island, South Kingston, RI, United States of America
| | | | | | | | | | | |
Collapse
|
46
|
The history of BCI: From a vision for the future to real support for personhood in people with locked-in syndrome. NEUROETHICS-NETH 2019. [DOI: 10.1007/s12152-019-09409-4] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
|
47
|
Chaudhary U, Pathak S, Birbaumer N. Response to: "Questioning the evidence for BCI-based communication in the complete locked-in state". PLoS Biol 2019; 17:e3000063. [PMID: 30958815 PMCID: PMC6453359 DOI: 10.1371/journal.pbio.3000063] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2018] [Accepted: 03/05/2019] [Indexed: 11/18/2022] Open
Affiliation(s)
- Ujwal Chaudhary
- Institute of Medical Psychology and Behavioral Neurobiology, Eberhrd-Karls University of Tuebingen, Tuebingen, Germany.,Wyss Center for Bio and Neuroengineering, Geneva, Switzerland
| | - Sudhir Pathak
- Learning Research and Development Center (LRDC), University of Pittsburgh, Pittsburgh, United States of America
| | - Niels Birbaumer
- Institute of Medical Psychology and Behavioral Neurobiology, Eberhrd-Karls University of Tuebingen, Tuebingen, Germany.,Wyss Center for Bio and Neuroengineering, Geneva, Switzerland
| |
Collapse
|
48
|
Spüler M. Questioning the evidence for BCI-based communication in the complete locked-in state. PLoS Biol 2019; 17:e2004750. [PMID: 30958814 PMCID: PMC6453399 DOI: 10.1371/journal.pbio.2004750] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2017] [Accepted: 03/05/2019] [Indexed: 11/19/2022] Open
Affiliation(s)
- Martin Spüler
- Department of Computer Engineering, Eberhard-Karls University Tübingen, Tübingen, Germany
- * E-mail:
| |
Collapse
|
49
|
Han CH, Kim YW, Kim DY, Kim SH, Nenadic Z, Im CH. Electroencephalography-based endogenous brain-computer interface for online communication with a completely locked-in patient. J Neuroeng Rehabil 2019; 16:18. [PMID: 30700310 PMCID: PMC6354345 DOI: 10.1186/s12984-019-0493-0] [Citation(s) in RCA: 35] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2018] [Accepted: 01/23/2019] [Indexed: 01/29/2023] Open
Abstract
Background Brain–computer interfaces (BCIs) have demonstrated the potential to provide paralyzed individuals with new means of communication, but an electroencephalography (EEG)-based endogenous BCI has never been successfully used for communication with a patient in a completely locked-in state (CLIS). Methods In this study, we investigated the possibility of using an EEG-based endogenous BCI paradigm for online binary communication by a patient in CLIS. A female patient in CLIS participated in this study. She had not communicated even with her family for more than one year with complete loss of motor function. Offline and online experiments were conducted to validate the feasibility of the proposed BCI system. In the offline experiment, we determined the best combination of mental tasks and the optimal classification strategy leading to the best performance. In the online experiment, we investigated whether our BCI system could be potentially used for real-time communication with the patient. Results An online classification accuracy of 87.5% was achieved when Riemannian geometry-based classification was applied to real-time EEG data recorded while the patient was performing one of two mental-imagery tasks for 5 s. Conclusions Our results suggest that an EEG-based endogenous BCI has the potential to be used for online communication with a patient in CLIS.
Collapse
Affiliation(s)
- Chang-Hee Han
- Department of Biomedical Engineering, Hanyang University, Seoul, 04763, South Korea
| | - Yong-Wook Kim
- Department of Biomedical Engineering, Hanyang University, Seoul, 04763, South Korea
| | - Do Yeon Kim
- Department of Biomedical Engineering, Hanyang University, Seoul, 04763, South Korea
| | - Seung Hyun Kim
- Department of Neurology, College of Medicine, Hanyang University, Seoul, 04763, South Korea
| | - Zoran Nenadic
- Department of Biomedical Engineering, University of California, Irvine, CA, 92697, USA
| | - Chang-Hwan Im
- Department of Biomedical Engineering, Hanyang University, Seoul, 04763, South Korea.
| |
Collapse
|
50
|
Abiri R, Borhani S, Sellers EW, Jiang Y, Zhao X. A comprehensive review of EEG-based brain–computer interface paradigms. J Neural Eng 2019; 16:011001. [DOI: 10.1088/1741-2552/aaf12e] [Citation(s) in RCA: 270] [Impact Index Per Article: 54.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
|