1
|
Wu X, Zhang D, Li G, Gao X, Metcalfe B, Chen L. Data augmentation for invasive brain-computer interfaces based on stereo-electroencephalography (SEEG). J Neural Eng 2024; 21:016026. [PMID: 38237174 DOI: 10.1088/1741-2552/ad200e] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2023] [Accepted: 01/18/2024] [Indexed: 02/23/2024]
Abstract
Objective.Deep learning is increasingly used for brain-computer interfaces (BCIs). However, the quantity of available data is sparse, especially for invasive BCIs. Data augmentation (DA) methods, such as generative models, can help to address this sparseness. However, all the existing studies on brain signals were based on convolutional neural networks and ignored the temporal dependence. This paper attempted to enhance generative models by capturing the temporal relationship from a time-series perspective.Approach. A conditional generative network (conditional transformer-based generative adversarial network (cTGAN)) based on the transformer model was proposed. The proposed method was tested using a stereo-electroencephalography (SEEG) dataset which was recorded from eight epileptic patients performing five different movements. Three other commonly used DA methods were also implemented: noise injection (NI), variational autoencoder (VAE), and conditional Wasserstein generative adversarial network with gradient penalty (cWGANGP). Using the proposed method, the artificial SEEG data was generated, and several metrics were used to compare the data quality, including visual inspection, cosine similarity (CS), Jensen-Shannon distance (JSD), and the effect on the performance of a deep learning-based classifier.Main results. Both the proposed cTGAN and the cWGANGP methods were able to generate realistic data, while NI and VAE outputted inferior samples when visualized as raw sequences and in a lower dimensional space. The cTGAN generated the best samples in terms of CS and JSD and outperformed cWGANGP significantly in enhancing the performance of a deep learning-based classifier (each of them yielding a significant improvement of 6% and 3.4%, respectively).Significance. This is the first time that DA methods have been applied to invasive BCIs based on SEEG. In addition, this study demonstrated the advantages of the model that preserves the temporal dependence from a time-series perspective.
Collapse
Affiliation(s)
- Xiaolong Wu
- The Centre for Autonomous Robotics (CENTAUR), Department of Electronic & Electrical Engineering, University of Bath, Bath, United Kingdom
| | - Dingguo Zhang
- The Centre for Autonomous Robotics (CENTAUR), Department of Electronic & Electrical Engineering, University of Bath, Bath, United Kingdom
| | - Guangye Li
- School of Mechanical Engineering, Shanghai Jiao Tong University, People's Republic of China
| | - Xin Gao
- The Centre for Autonomous Robotics (CENTAUR), Department of Electronic & Electrical Engineering, University of Bath, Bath, United Kingdom
| | - Benjamin Metcalfe
- The Centre for Autonomous Robotics (CENTAUR), Department of Electronic & Electrical Engineering, University of Bath, Bath, United Kingdom
| | - Liang Chen
- Liang Chen is with Huashan Hospital, Fudan University, People's Republic of China
| |
Collapse
|
2
|
Branco MP, Geukes SH, Aarnoutse EJ, Ramsey NF, Vansteensel MJ. Nine decades of electrocorticography: A comparison between epidural and subdural recordings. Eur J Neurosci 2023; 57:1260-1288. [PMID: 36843389 DOI: 10.1111/ejn.15941] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2022] [Revised: 02/10/2023] [Accepted: 02/18/2023] [Indexed: 02/28/2023]
Abstract
In recent years, electrocorticography (ECoG) has arisen as a neural signal recording tool in the development of clinically viable neural interfaces. ECoG electrodes are generally placed below the dura mater (subdural) but can also be placed on top of the dura (epidural). In deciding which of these modalities best suits long-term implants, complications and signal quality are important considerations. Conceptually, epidural placement may present a lower risk of complications as the dura is left intact but also a lower signal quality due to the dura acting as a signal attenuator. The extent to which complications and signal quality are affected by the dura, however, has been a matter of debate. To improve our understanding of the effects of the dura on complications and signal quality, we conducted a literature review. We inventorized the effect of the dura on signal quality, decodability and longevity of acute and chronic ECoG recordings in humans and non-human primates. Also, we compared the incidence and nature of serious complications in studies that employed epidural and subdural ECoG. Overall, we found that, even though epidural recordings exhibit attenuated signal amplitude over subdural recordings, particularly for high-density grids, the decodability of epidural recorded signals does not seem to be markedly affected. Additionally, we found that the nature of serious complications was comparable between epidural and subdural recordings. These results indicate that both epidural and subdural ECoG may be suited for long-term neural signal recordings, at least for current generations of clinical and high-density ECoG grids.
Collapse
Affiliation(s)
- Mariana P Branco
- Department of Neurology and Neurosurgery, University Medical Center Utrecht Brain Center, Utrecht University, Utrecht, The Netherlands
| | - Simon H Geukes
- Department of Neurology and Neurosurgery, University Medical Center Utrecht Brain Center, Utrecht University, Utrecht, The Netherlands
| | - Erik J Aarnoutse
- Department of Neurology and Neurosurgery, University Medical Center Utrecht Brain Center, Utrecht University, Utrecht, The Netherlands
| | - Nick F Ramsey
- Department of Neurology and Neurosurgery, University Medical Center Utrecht Brain Center, Utrecht University, Utrecht, The Netherlands
| | - Mariska J Vansteensel
- Department of Neurology and Neurosurgery, University Medical Center Utrecht Brain Center, Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
3
|
Fischer B, Schander A, Kreiter AK, Lang W, Wegener D. Visual epidural field potentials possess high functional specificity in single trials. J Neurophysiol 2019; 122:1634-1648. [PMID: 31412218 DOI: 10.1152/jn.00510.2019] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Recordings of epidural field potentials (EFPs) allow neuronal activity to be acquired over a large region of cortical tissue with minimal invasiveness. Because electrodes are placed on top of the dura and do not enter the neuronal tissue, EFPs offer intriguing options for both clinical and basic science research. On the other hand, EFPs represent the integrated activity of larger neuronal populations and possess a higher trial-by-trial variability and a reduced signal-to-noise ratio due the additional barrier of the dura. It is thus unclear whether and to what extent EFPs have sufficient spatial selectivity to allow for conclusions about the underlying functional cortical architecture, and whether single EFP trials provide enough information on the short timescales relevant for many clinical and basic neuroscience purposes. We used the high spatial resolution of primary visual cortex to address these issues and investigated the extent to which very short EFP traces allow reliable decoding of spatial information. We briefly presented different visual objects at one of nine closely adjacent locations and recorded neuronal activity with a high-density epidural multielectrode array in three macaque monkeys. With the use of receiver operating characteristics (ROC) to identify the most informative data, machine-learning algorithms provided close-to-perfect classification rates for all 27 stimulus conditions. A binary classifier applying a simple max function on ROC-selected data further showed that single trials might be classified with 100% performance even without advanced offline classifiers. Thus, although highly variable, EFPs constitute an extremely valuable source of information and offer new perspectives for minimally invasive recording of large-scale networks.NEW & NOTEWORTHY Epidural field potential (EFP) recordings provide a minimally invasive approach to investigate large-scale neural networks, but little is known about whether they possess the required specificity for basic and clinical neuroscience. By making use of the spatial selectivity of primary visual cortex, we show that single-trial information can be decoded with close-to-perfect performance, even without using advanced classifiers and based on very few data. This labels EFPs as a highly attractive and widely usable signal.
Collapse
Affiliation(s)
- Benjamin Fischer
- Brain Research Institute, Center for Cognitive Sciences, University of Bremen, Bremen, Germany
| | - Andreas Schander
- Institute for Microsensors, -Actuators, and -Systems, University of Bremen, Bremen, Germany
| | - Andreas K Kreiter
- Brain Research Institute, Center for Cognitive Sciences, University of Bremen, Bremen, Germany
| | - Walter Lang
- Institute for Microsensors, -Actuators, and -Systems, University of Bremen, Bremen, Germany
| | - Detlef Wegener
- Brain Research Institute, Center for Cognitive Sciences, University of Bremen, Bremen, Germany
| |
Collapse
|
4
|
John SE, Opie NL, Wong YT, Rind GS, Ronayne SM, Gerboni G, Bauquier SH, O'Brien TJ, May CN, Grayden DB, Oxley TJ. Signal quality of simultaneously recorded endovascular, subdural and epidural signals are comparable. Sci Rep 2018; 8:8427. [PMID: 29849104 PMCID: PMC5976775 DOI: 10.1038/s41598-018-26457-7] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2017] [Accepted: 05/10/2018] [Indexed: 02/07/2023] Open
Abstract
Recent work has demonstrated the feasibility of minimally-invasive implantation of electrodes into a cortical blood vessel. However, the effect of the dura and blood vessel on recording signal quality is not understood and may be a critical factor impacting implementation of a closed-loop endovascular neuromodulation system. The present work compares the performance and recording signal quality of a minimally-invasive endovascular neural interface with conventional subdural and epidural interfaces. We compared bandwidth, signal-to-noise ratio, and spatial resolution of recorded cortical signals using subdural, epidural and endovascular arrays four weeks after implantation in sheep. We show that the quality of the signals (bandwidth and signal-to-noise ratio) of the endovascular neural interface is not significantly different from conventional neural sensors. However, the spatial resolution depends on the array location and the frequency of recording. We also show that there is a direct correlation between the signal-noise-ratio and classification accuracy, and that decoding accuracy is comparable between electrode arrays. These results support the consideration for use of an endovascular neural interface in a clinical trial of a novel closed-loop neuromodulation technology.
Collapse
Affiliation(s)
- Sam E John
- Department of Biomedical Engineering, The University of Melbourne, Parkville, Australia. .,Vascular Bionics Laboratory, Department of Medicine, Royal Melbourne Hospital, (RMH), The University of Melbourne, Parkville, Australia. .,Florey Institute of Neuroscience and Mental Health, Parkville, Australia. .,SmartStent Pty Ltd, Parkville, Australia.
| | - Nicholas L Opie
- Vascular Bionics Laboratory, Department of Medicine, Royal Melbourne Hospital, (RMH), The University of Melbourne, Parkville, Australia.,Florey Institute of Neuroscience and Mental Health, Parkville, Australia.,SmartStent Pty Ltd, Parkville, Australia
| | - Yan T Wong
- Department of Biomedical Engineering, The University of Melbourne, Parkville, Australia.,Department of Physiology and Department of Electrical and Computer Systems Engineering, Monash University, Clayton, Australia
| | - Gil S Rind
- Vascular Bionics Laboratory, Department of Medicine, Royal Melbourne Hospital, (RMH), The University of Melbourne, Parkville, Australia.,Florey Institute of Neuroscience and Mental Health, Parkville, Australia.,SmartStent Pty Ltd, Parkville, Australia
| | - Stephen M Ronayne
- Vascular Bionics Laboratory, Department of Medicine, Royal Melbourne Hospital, (RMH), The University of Melbourne, Parkville, Australia.,Florey Institute of Neuroscience and Mental Health, Parkville, Australia.,SmartStent Pty Ltd, Parkville, Australia
| | - Giulia Gerboni
- Department of Biomedical Engineering, The University of Melbourne, Parkville, Australia.,Vascular Bionics Laboratory, Department of Medicine, Royal Melbourne Hospital, (RMH), The University of Melbourne, Parkville, Australia.,Florey Institute of Neuroscience and Mental Health, Parkville, Australia
| | - Sebastien H Bauquier
- Department of Veterinary Science, The University of Melbourne, Werribee, Australia
| | - Terence J O'Brien
- Vascular Bionics Laboratory, Department of Medicine, Royal Melbourne Hospital, (RMH), The University of Melbourne, Parkville, Australia.,Florey Institute of Neuroscience and Mental Health, Parkville, Australia
| | - Clive N May
- Florey Institute of Neuroscience and Mental Health, Parkville, Australia
| | - David B Grayden
- Department of Biomedical Engineering, The University of Melbourne, Parkville, Australia.,Centre for Neural Engineering, The University of Melbourne, Carlton, Australia
| | - Thomas J Oxley
- Vascular Bionics Laboratory, Department of Medicine, Royal Melbourne Hospital, (RMH), The University of Melbourne, Parkville, Australia.,Florey Institute of Neuroscience and Mental Health, Parkville, Australia.,SmartStent Pty Ltd, Parkville, Australia
| |
Collapse
|
5
|
Kellmeyer P, Grosse-Wentrup M, Schulze-Bonhage A, Ziemann U, Ball T. Electrophysiological correlates of neurodegeneration in motor and non-motor brain regions in amyotrophic lateral sclerosis-implications for brain-computer interfacing. J Neural Eng 2018; 15:041003. [PMID: 29676287 DOI: 10.1088/1741-2552/aabfa5] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
OBJECTIVE For patients with amyotrophic lateral sclerosis (ALS) who are suffering from severe communication or motor problems, brain-computer interfaces (BCIs) can improve the quality of life and patient autonomy. However, current BCI systems are not as widely used as their potential and patient demand would let assume. This underutilization is a result of technological as well as user-based limitations but also of the comparatively poor performance of currently existing BCIs in patients with late-stage ALS, particularly in the locked-in state. APPROACH Here we review a broad range of electrophysiological studies in ALS patients with the aim to identify electrophysiological correlates of ALS-related neurodegeneration in motor and non-motor brain regions in to better understand potential neurophysiological limitations of current BCI systems for ALS patients. To this end we analyze studies in ALS patients that investigated basic sensory evoked potentials, resting-state and task-based paradigms using electroencephalography or electrocorticography for basic research purposes as well as for brain-computer interfacing. Main results and significance. Our review underscores that, similarly to mounting evidence from neuroimaging and neuropathology, electrophysiological measures too indicate neurodegeneration in non-motor areas in ALS. Furthermore, we identify an unexpected gap of basic and advanced electrophysiological studies in late-stage ALS patients, particularly in the locked-in state. We propose a research strategy on how to fill this gap in order to improve the design and performance of future BCI systems for this patient group.
Collapse
Affiliation(s)
- Philipp Kellmeyer
- Translational Neurotechnology Lab, Department of Neurosurgery, Medical Center-University of Freiburg, Freiburg im Breisgau, Germany. Cluster of Excellence BrainLinks-BrainTools, University of Freiburg, Freiburg im Breisgau, Germany
| | | | | | | | | |
Collapse
|
6
|
Lebedev MA, Nicolelis MAL. Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation. Physiol Rev 2017; 97:767-837. [PMID: 28275048 DOI: 10.1152/physrev.00027.2016] [Citation(s) in RCA: 233] [Impact Index Per Article: 33.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2023] Open
Abstract
Brain-machine interfaces (BMIs) combine methods, approaches, and concepts derived from neurophysiology, computer science, and engineering in an effort to establish real-time bidirectional links between living brains and artificial actuators. Although theoretical propositions and some proof of concept experiments on directly linking the brains with machines date back to the early 1960s, BMI research only took off in earnest at the end of the 1990s, when this approach became intimately linked to new neurophysiological methods for sampling large-scale brain activity. The classic goals of BMIs are 1) to unveil and utilize principles of operation and plastic properties of the distributed and dynamic circuits of the brain and 2) to create new therapies to restore mobility and sensations to severely disabled patients. Over the past decade, a wide range of BMI applications have emerged, which considerably expanded these original goals. BMI studies have shown neural control over the movements of robotic and virtual actuators that enact both upper and lower limb functions. Furthermore, BMIs have also incorporated ways to deliver sensory feedback, generated from external actuators, back to the brain. BMI research has been at the forefront of many neurophysiological discoveries, including the demonstration that, through continuous use, artificial tools can be assimilated by the primate brain's body schema. Work on BMIs has also led to the introduction of novel neurorehabilitation strategies. As a result of these efforts, long-term continuous BMI use has been recently implicated with the induction of partial neurological recovery in spinal cord injury patients.
Collapse
|