1
|
Jin J, Chen W, Xu R, Liang W, Wu X, He X, Wang X, Cichocki A. Multiscale Spatial-Temporal Feature Fusion Neural Network for Motor Imagery Brain-Computer Interfaces. IEEE J Biomed Health Inform 2025; 29:198-209. [PMID: 39352826 DOI: 10.1109/jbhi.2024.3472097] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/04/2024]
Abstract
Motor imagery, one of the main brain-computer interface (BCI) paradigms, has been extensively utilized in numerous BCI applications, such as the interaction between disabled people and external devices. Precise decoding, one of the most significant aspects of realizing efficient and stable interaction, has received a great deal of intensive research. However, the current decoding methods based on deep learning are still dominated by single-scale serial convolution, which leads to insufficient extraction of abundant information from motor imagery signals. To overcome such challenges, we propose a new end-to-end convolutional neural network based on multiscale spatial-temporal feature fusion (MSTFNet) for EEG classification of motor imagery. The architecture of MSTFNet consists of four distinct modules: feature enhancement module, multiscale temporal feature extraction module, spatial feature extraction module and feature fusion module, with the latter being further divided into the depthwise separable convolution block and efficient channel attention block. Moreover, we implement a straightforward yet potent data augmentation strategy to bolster the performance of MSTFNet significantly. To validate the performance of MSTFNet, we conduct cross-session experiments and leave-one-subject-out experiments. The cross-session experiment is conducted across two public datasets and one laboratory dataset. On the public datasets of BCI Competition IV 2a and BCI Competition IV 2b, MSTFNet achieves classification accuracies of 83.62% and 89.26%, respectively. On the laboratory dataset, MSTFNet achieves 86.68% classification accuracy. Besides, the leave-one-subject-out experiment is performed on the BCI Competition IV 2a dataset, and MSTFNet achieves 66.31% classification accuracy. These experimental results outperform several state-of-the-art methodologies, indicate the proposed MSTFNet's robust capability in decoding EEG signals associated with motor imagery.
Collapse
|
2
|
King BJ, Read GJM, Salmon PM. Prospectively identifying risks and controls for advanced brain-computer interfaces: A Networked Hazard Analysis and Risk Management System (Net-HARMS) approach. APPLIED ERGONOMICS 2025; 122:104382. [PMID: 39265503 DOI: 10.1016/j.apergo.2024.104382] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/29/2024] [Revised: 08/26/2024] [Accepted: 09/04/2024] [Indexed: 09/14/2024]
Abstract
The introduction of advanced digital technologies continues to increase system complexity and introduce risks, which must be proactively identified and managed to support system resilience. Brain-computer interfaces (BCIs) are one such technology; however, the risks arising from broad societal use of the technology have yet to be identified and controlled. This study applied a structured systems thinking-based risk assessment method to prospectively identify risks and risk controls for a hypothetical future BCI system lifecycle. The application of the Networked Hazard Analysis and Risk Management System (Net-HARMS) method identified over 800 risks throughout the BCI system lifecycle, from BCI development and regulation through to BCI use, maintenance, and decommissioning. High-criticality risk themes include the implantation and degradation of unsafe BCIs, unsolicited brain stimulation, incorrect signals being sent to safety-critical technologies, and insufficiently supported BCI users. Over 600 risk controls were identified that could be implemented to support system safety and performance resilience. Overall, many highly-impactful BCI system safety and performance risks may arise throughout the BCI system lifecycle and will require collaborative efforts from a wide range of BCI stakeholders to adequately control. Whilst some of the identified controls are practical, work is required to develop a more systematic set of controls to best support the design of a resilient sociotechnical BCI system.
Collapse
Affiliation(s)
- Brandon J King
- Centre for Human Factors and Sociotechnical Systems, University of the Sunshine Coast, ML 47, Locked Bag 4, Maroochydore DC, Queensland, 4558, Australia; Centre for Human Factors and Sociotechnical Systems, University of the Sunshine Coast, Australia.
| | - Gemma J M Read
- Centre for Human Factors and Sociotechnical Systems, University of the Sunshine Coast, Australia; School of Health, University of the Sunshine Coast, Australia. https://twitter.com/gemma_read
| | - Paul M Salmon
- Centre for Human Factors and Sociotechnical Systems, University of the Sunshine Coast, Australia. https://twitter.com/DrPaulSalmon
| |
Collapse
|
3
|
Opancina V, Sebek V, Janjic V. Advanced neuroimaging and criminal interrogation in lie detection. Open Med (Wars) 2024; 19:20241032. [PMID: 39247439 PMCID: PMC11377981 DOI: 10.1515/med-2024-1032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2024] [Revised: 07/28/2024] [Accepted: 08/16/2024] [Indexed: 09/10/2024] Open
Abstract
Hidden information is the key to many security issues. If there is a reliable method to determine whether someone withholds information, many issues of this type can be resolved. However, until now, no method has proven to be reliable, but technical discoveries in the field of neuroimaging have caused a surge of new research in this area. Many neuroimaging techniques can be used, but functional magnetic resonance is the newest method, and its use in extracting and evaluating information from subjects could be the most significant, given that it records brain states in parallel with current mental activity/behavior, enabling the establishment of correlational links between them. Because the brain state displayed during fMRI imaging is the dependent variable measured during stimulus/task condition manipulation, it is necessary to use fMRI data in combination with complementary criminal interrogation techniques to gather information. This could be particularly important when standard interrogational techniques are not enough in order to preserve the common good, especially in "ticking bomb" situations. In this study, we review aspects of the possibility of utilizing advanced neuroimaging in combination with criminal interrogation in cases of serious criminal acts that threaten public safety.
Collapse
Affiliation(s)
- Valentina Opancina
- Department of Radiology, Faculty of Medical Sciences, University of Kragujevac, Kragujevac, Serbia
- University Clinical Center Kragujevac, Kragujevac, Serbia
| | - Vladimir Sebek
- Department of Criminalistics, Faculty of law, University of Kragujevac, Kragujevac, Serbia
- Regional police directorate of Kragujevac, Republic of Serbia, Police Directorate, Ministry of interior, Kragujevac, Serbia
| | - Vladimir Janjic
- Department of Communication Skills, Ethics and Psychology, Faculty of Medical Sciences, University of Kragujevac, Kragujevac, Serbia
- University Clinical Center Kragujevac, Kragujevac, Serbia
| |
Collapse
|
4
|
King BJ, Read GJM, Salmon PM. Identifying risk controls for future advanced brain-computer interfaces: A prospective risk assessment approach using work domain analysis. APPLIED ERGONOMICS 2023; 111:104028. [PMID: 37148587 DOI: 10.1016/j.apergo.2023.104028] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/27/2022] [Revised: 04/13/2023] [Accepted: 04/13/2023] [Indexed: 05/08/2023]
Abstract
Brain-computer interface (BCI) technologies are progressing rapidly and may eventually be implemented widely within society, yet their risks have arguably not yet been comprehensively identified, nor understood. This study analysed an anticipated invasive BCI system lifecycle to identify the individual, organisational, and societal risks associated with BCIs, and controls that could be used to mitigate or eliminate these risks. A BCI system lifecycle work domain analysis model was developed and validated with 10 subject matter experts. The model was subsequently used to undertake a systems thinking-based risk assessment approach to identify risks that could emerge when functions are either undertaken sub-optimally or not undertaken at all. Eighteen broad risk themes were identified that could negatively impact the BCI system lifecycle in a variety of unique ways, while a larger number of controls for these risks were also identified. The most concerning risks included inadequate regulation of BCI technologies and inadequate training of BCI stakeholders, such as users and clinicians. In addition to specifying a practical set of risk controls to inform BCI device design, manufacture, adoption, and utilisation, the results demonstrate the complexity involved in managing BCI risks and suggests that a system-wide coordinated response is required. Future research is required to evaluate the comprehensiveness of the identified risks and the practicality of implementing the risk controls.
Collapse
Affiliation(s)
- Brandon J King
- Centre for Human Factors and Sociotechnical Systems, University of the Sunshine Coast, Australia.
| | - Gemma J M Read
- Centre for Human Factors and Sociotechnical Systems, University of the Sunshine Coast, Australia; School of Health, University of the Sunshine Coast, Australia. https://twitter.com/gemma_read
| | - Paul M Salmon
- Centre for Human Factors and Sociotechnical Systems, University of the Sunshine Coast, Australia. https://twitter.com/DrPaulSalmon
| |
Collapse
|
5
|
Kosal M, Putney J. Neurotechnology and international security: Predicting commercial and military adoption of brain-computer interfaces (BCIs) in the United States and China. Politics Life Sci 2023; 42:81-103. [PMID: 37140225 DOI: 10.1017/pls.2022.2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/05/2023]
Abstract
In the past decade, international actors have launched "brain projects" or "brain initiatives." One of the emerging technologies enabled by these publicly funded programs is brain-computer interfaces (BCIs), which are devices that allow communication between the brain and external devices like a prosthetic arm or a keyboard. BCIs are poised to have significant impacts on public health, society, and national security. This research presents the first analytical framework that attempts to predict the dissemination of neurotechnologies to both the commercial and military sectors in the United States and China. While China started its project later with less funding, we find that it has other advantages that make earlier adoption more likely. We also articulate national security risks implicit in later adoption, including the inability to set international ethical and legal norms for BCI use, especially in wartime operating environments, and data privacy risks for citizens who use technology developed by foreign actors.
Collapse
|
6
|
Recognition of EEG based on Improved Black Widow Algorithm optimized SVM. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104454] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/02/2022]
|
7
|
Liu S, Zhang D, Liu Z, Liu M, Ming Z, Liu T, Suo D, Funahashi S, Yan T. Review of brain–computer interface based on steady‐state visual evoked potential. BRAIN SCIENCE ADVANCES 2022. [DOI: 10.26599/bsa.2022.9050022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/03/2022] Open
Abstract
The brain–computer interface (BCI) technology has received lots of attention in the field of scientific research because it can help disabled people improve their quality of life. Steady‐state visual evoked potential (SSVEP) is the most researched BCI experimental paradigm, which offers the advantages of high signal‐to‐noise ratio and short training‐time requirement by users. In a complete BCI system, the two most critical components are the experimental paradigm and decoding algorithm. However, a systematic combination of the SSVEP experimental paradigm and decoding algorithms is missing in existing studies. In the present study, the transient visual evoked potential, SSVEP, and various improved SSVEP paradigms are compared and analyzed, and the problems and development bottlenecks in the experimental paradigm are finally pointed out. Subsequently, the canonical correlation analysis and various improved decoding algorithms are introduced, and the opportunities and challenges of the SSVEP decoding algorithm are discussed.
Collapse
Affiliation(s)
- Siyu Liu
- School of Life Science, Beijing Institute of Technology, Beijing 100081, China
| | - Deyu Zhang
- School of Mechanical and Electrical Engineering, Beijing Institute of Technology, Beijing 100081, China
| | - Ziyu Liu
- School of Life Science, Beijing Institute of Technology, Beijing 100081, China
| | - Mengzhen Liu
- School of Mechanical and Electrical Engineering, Beijing Institute of Technology, Beijing 100081, China
| | - Zhiyuan Ming
- School of Mechanical and Electrical Engineering, Beijing Institute of Technology, Beijing 100081, China
| | - Tiantian Liu
- School of Life Science, Beijing Institute of Technology, Beijing 100081, China
| | - Dingjie Suo
- School of Life Science, Beijing Institute of Technology, Beijing 100081, China
| | - Shintaro Funahashi
- Advanced Research Institute of Multidisciplinary Science, Beijing Institute of Technology, Beijing 100981, China
- Kyoto University, Yoshida‐honmachi 606‐8501, Kyoto‐Shi, Japan
| | - Tianyi Yan
- School of Life Science, Beijing Institute of Technology, Beijing 100081, China
| |
Collapse
|
8
|
Promise for Personalized Diagnosis? Assessing the Precision of Wireless Consumer-Grade Electroencephalography across Mental States. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12136430] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/27/2023]
Abstract
In the last decade there has been significant growth in the interest and application of using EEG (electroencephalography) outside of laboratory as well as in medical and clinical settings, for more ecological and mobile applications. However, for now such applications have mainly included military, educational, cognitive enhancement, and consumer-based games. Given the monetary and ecological advantages, consumer-grade EEG devices such as the Emotiv EPOC have emerged, however consumer-grade devices make certain compromises of data quality in order to become affordable and easy to use. The goal of this study was to investigate the reliability and accuracy of EPOC as compared to a research-grade device, Brainvision. To this end, we collected data from participants using both devices during three distinct cognitive tasks designed to elicit changes in arousal, valence, and cognitive load: namely, Affective Norms for English Words, International Affective Picture System, and the n-Back task. Our design and analytical strategies followed an ideographic person-level approach (electrode-wise analysis of vincentized repeated measures). We aimed to assess how well the Emotiv could differentiate between mental states using an Event-Related Band Power approach and EEG features such as amplitude and power, as compared to Brainvision. The Emotiv device was able to differentiate mental states during these tasks to some degree, however it was generally poorer than Brainvision, with smaller effect sizes. The Emotiv may be used with reasonable reliability and accuracy in ecological settings and in some clinical contexts (for example, for training professionals), however Brainvision or other, equivalent research-grade devices are still recommended for laboratory or medical based applications.
Collapse
|
9
|
Chang L, Wang R, Zhang Y. Decoding SSVEP patterns from EEG via multivariate variational mode decomposition-informed canonical correlation analysis. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103209] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
10
|
Bhattacharyya S, Hayashibe M. An Optimal Transport Based Transferable System for Detection of Erroneous Somato-Sensory Feedback from Neural Signals. Brain Sci 2021; 11:1393. [PMID: 34827392 PMCID: PMC8615878 DOI: 10.3390/brainsci11111393] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2021] [Revised: 10/15/2021] [Accepted: 10/19/2021] [Indexed: 11/26/2022] Open
Abstract
This study is aimed at the detection of single-trial feedback, perceived as erroneous by the user, using a transferable classification system while conducting a motor imagery brain-computer interfacing (BCI) task. The feedback received by the users are relayed from a functional electrical stimulation (FES) device and hence are somato-sensory in nature. The BCI system designed for this study activates an electrical stimulator placed on the left hand, right hand, left foot, and right foot of the user. Trials containing erroneous feedback can be detected from the neural signals in form of the error related potential (ErrP). The inclusion of neuro-feedback during the experiments indicated the possibility that ErrP signals can be evoked when the participant perceives an error from the feedback. Hence, to detect such feedback using ErrP, a transferable (offline) decoder based on optimal transport theory is introduced herein. The offline system detects single-trial erroneous trials from the feedback period of an online neuro-feedback BCI system. The results of the FES-based feedback BCI system were compared to a similar visual-based (VIS) feedback system. Using our framework, the error detector systems for both the FES and VIS feedback paradigms achieved an F1-score of 92.66% and 83.10%, respectively, and are significantly superior to a comparative system where an optimal transport was not used. It is expected that this form of transferable and automated error detection system compounded with a motor imagery system will augment the performance of a BCI and provide a better BCI-based neuro-rehabilitation protocol that has an error control mechanism embedded into it.
Collapse
Affiliation(s)
- Saugat Bhattacharyya
- School of Computing, Engineering and Intelligent Systems, Ulster University, Magee Campus, Londonderry BT48 7JL, UK
| | - Mitsuhiro Hayashibe
- Department of Robotics, Tohoku University, Sendai 980-8579, Japan;
- Department of Biomedical Engineering, Tohoku University, Sendai 980-8579, Japan
| |
Collapse
|
11
|
Task Transfer Learning for EEG Classification in Motor Imagery-Based BCI System. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2021; 2020:6056383. [PMID: 33381220 PMCID: PMC7755477 DOI: 10.1155/2020/6056383] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/17/2020] [Revised: 06/25/2020] [Accepted: 07/27/2020] [Indexed: 11/30/2022]
Abstract
The motor-imagery brain-computer interface system (MI-BCI) has a board prospect for development. However, long calibration time and lack of enough MI commands limit its use in practice. In order to enlarge the command set, we add the combinations of traditional MI commands as new commands into the command set. We also design an algorithm based on transfer learning so as to decrease the calibration time for collecting EEG signal and training model. We create feature extractor based on data from traditional commands and transfer patterns through the data from new commands. Through the comparison of the average accuracy between our algorithm and traditional algorithms and the visualization of spatial patterns in our algorithm, we find that the accuracy of our algorithm is much higher than traditional algorithms, especially as for the low-quality datasets. Besides, the visualization of spatial patterns is meaningful. The algorithm based on transfer learning takes the advantage of the information from source data. We enlarge the command set while shortening the calibration time, which is of significant importance to the MI-BCI application.
Collapse
|
12
|
O'Brien JT, Nelson C. Assessing the Risks Posed by the Convergence of Artificial Intelligence and Biotechnology. Health Secur 2020; 18:219-227. [PMID: 32559154 PMCID: PMC7310294 DOI: 10.1089/hs.2019.0122] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2019] [Revised: 03/04/2020] [Accepted: 04/29/2020] [Indexed: 12/22/2022] Open
Abstract
Rapid developments are currently taking place in the fields of artificial intelligence (AI) and biotechnology, and applications arising from the convergence of these 2 fields are likely to offer immense opportunities that could greatly benefit human health and biosecurity. The combination of AI and biotechnology could potentially lead to breakthroughs in precision medicine, improved biosurveillance, and discovery of novel medical countermeasures as well as facilitate a more effective public health emergency response. However, as is the case with many preceding transformative technologies, new opportunities often present new risks in parallel. Understanding the current and emerging risks at the intersection of AI and biotechnology is crucial for health security specialists and unlikely to be achieved by examining either field in isolation. Uncertainties multiply as technologies merge, showcasing the need to identify robust assessment frameworks that could adequately analyze the risk landscape emerging at the convergence of these 2 domains.This paper explores the criteria needed to assess risks associated with Artificial intelligence and biotechnology and evaluates 3 previously published risk assessment frameworks. After highlighting their strengths and limitations and applying to relevant Artificial intelligence and biotechnology examples, the authors suggest a hybrid framework with recommendations for future approaches to risk assessment for convergent technologies.
Collapse
Affiliation(s)
- John T. O'Brien
- John T. O'Brien, MS, is a Research Associate, Bipartisan Commission on Biodefense, Washington, DC
| | - Cassidy Nelson
- Cassidy Nelson, MBBS, MPH, is a Research Scholar, Future of Humanity Institute, University of Oxford, Oxford, UK
| |
Collapse
|
13
|
Chari A, Budhdeo S, Sparks R, Barone DG, Marcus HJ, Pereira EAC, Tisdall MM. Brain-Machine Interfaces: The Role of the Neurosurgeon. World Neurosurg 2020; 146:140-147. [PMID: 33197630 DOI: 10.1016/j.wneu.2020.11.028] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2020] [Revised: 11/04/2020] [Accepted: 11/05/2020] [Indexed: 12/26/2022]
Abstract
Neurotechnology is set to expand rapidly in the coming years as technological innovations in hardware and software are translated to the clinical setting. Given our unique access to patients with neurologic disorders, expertise with which to guide appropriate treatments, and technical skills to implant brain-machine interfaces (BMIs), neurosurgeons have a key role to play in the progress of this field. We outline the current state and key challenges in this rapidly advancing field, including implant technology, implant recipients, implantation methodology, implant function, and ethical, regulatory, and economic considerations. Our key message is to encourage the neurosurgical community to proactively engage in collaborating with other health care professionals, engineers, scientists, ethicists, and regulators in tackling these issues. By doing so, we will equip ourselves with the skills and expertise to drive the field forward and avoid being mere technicians in an industry driven by those around us.
Collapse
Affiliation(s)
- Aswin Chari
- Developmental Neurosciences, Great Ormond Street Institute of Child Health, University College London, London, United Kingdom; Department of Neurosurgery, Great Ormond Street Hospital, London, United Kingdom.
| | - Sanjay Budhdeo
- Department for Clinical and Movement Neurosciences, Queen Square Institute of Neurology, University College London, London, United Kingdom; Department of Neurology, National Hospital for Neurology and Neurosurgery, London, United Kingdom; OwkinInc, New York, New York, USA
| | - Rachel Sparks
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom
| | - Damiano G Barone
- Division of Neurosurgery, Department of Clinical Neurosciences, University of Cambridge, Cambridge, United Kingdom
| | - Hani J Marcus
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, United Kingdom; Wellcome EPSRC Centre for Interventional and Surgical Sciences, University College London, London, United Kingdom
| | - Erlick A C Pereira
- Neurosciences Research Centre, Molecular and Clinical Sciences Research Institute, St George's, University of London, United Kingdom
| | - Martin M Tisdall
- Developmental Neurosciences, Great Ormond Street Institute of Child Health, University College London, London, United Kingdom; Department of Neurosurgery, Great Ormond Street Hospital, London, United Kingdom
| |
Collapse
|