1
|
Iwama S, Takemi M, Eguchi R, Hirose R, Morishige M, Ushiba J. Two common issues in synchronized multimodal recordings with EEG: Jitter and latency. Neurosci Res 2024; 203:1-7. [PMID: 38141782 DOI: 10.1016/j.neures.2023.12.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Revised: 11/19/2023] [Accepted: 12/13/2023] [Indexed: 12/25/2023]
Abstract
Multimodal recording using electroencephalogram (EEG) and other biological signals (e.g., muscle activities, eye movement, pupil diameters, or body kinematics data) is ubiquitous in human neuroscience research. However, the precise time alignment of multiple data from heterogeneous sources (i.e., devices) is often arduous due to variable recording parameters of commercially available research devices and complex experimental setups. In this review, we introduced the versatility of a Lab Streaming Layer (LSL)-based application that can overcome two common issues in measuring multimodal data: jitter and latency. We discussed the issues of jitter and latency in multimodal recordings and the benefits of time-synchronization when recording with multiple devices. In addition, a computer simulation was performed to highlight how the millisecond-order jitter readily affects the signal-to-noise ratio of the electrophysiological outcome. Together, we argue that the LSL-based system can be used for research requiring precise time-alignment of datasets. Studies that detect stimulus-induced transient neural responses or test hypotheses regarding temporal relationships of different functional aspects with multimodal data would benefit most from LSL-based systems.
Collapse
Affiliation(s)
- Seitaro Iwama
- Department of Biosciences and Informatics, Faculty of Science and Technology, Keio University, Japan
| | - Mitsuaki Takemi
- Graduate School of Science and Technology, Keio University, Japan; Japan Science and Technology Agency PRESTO, Japan
| | - Ryo Eguchi
- Graduate School of Science and Technology, Keio University, Japan
| | - Ryotaro Hirose
- Graduate School of Science and Technology, Keio University, Japan
| | - Masumi Morishige
- Graduate School of Science and Technology, Keio University, Japan
| | - Junichi Ushiba
- Department of Biosciences and Informatics, Faculty of Science and Technology, Keio University, Japan.
| |
Collapse
|
2
|
Zhang DW, Johnstone SJ, Sauce B, Arns M, Sun L, Jiang H. Remote neurocognitive interventions for attention-deficit/hyperactivity disorder - Opportunities and challenges. Prog Neuropsychopharmacol Biol Psychiatry 2023; 127:110802. [PMID: 37257770 DOI: 10.1016/j.pnpbp.2023.110802] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Revised: 05/23/2023] [Accepted: 05/24/2023] [Indexed: 06/02/2023]
Abstract
Improving neurocognitive functions through remote interventions has been a promising approach to developing new treatments for attention-deficit/hyperactivity disorder (AD/HD). Remote neurocognitive interventions may address the shortcomings of the current prevailing pharmacological therapies for AD/HD, e.g., side effects and access barriers. Here we review the current options for remote neurocognitive interventions to reduce AD/HD symptoms, including cognitive training, EEG neurofeedback training, transcranial electrical stimulation, and external cranial nerve stimulation. We begin with an overview of the neurocognitive deficits in AD/HD to identify the targets for developing interventions. The role of neuroplasticity in each intervention is then highlighted due to its essential role in facilitating neuropsychological adaptations. Following this, each intervention type is discussed in terms of the critical details of the intervention protocols, the role of neuroplasticity, and the available evidence. Finally, we offer suggestions for future directions in terms of optimizing the existing intervention protocols and developing novel protocols.
Collapse
Affiliation(s)
- Da-Wei Zhang
- Department of Psychology/Center for Place-Based Education, Yangzhou University, Yangzhou, China; Department of Psychology, Monash University Malaysia, Bandar Sunway, Malaysia.
| | - Stuart J Johnstone
- School of Psychology, University of Wollongong, Wollongong, Australia; Brain & Behaviour Research Institute, University of Wollongong, Australia
| | - Bruno Sauce
- Department of Biological Psychology, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| | - Martijn Arns
- Research Institute Brainclinics, Brainclinics Foundation, Nijmegen, Netherlands; Department of Experimental Psychology, Utrecht University, Utrecht, Netherlands; NeuroCare Group, Nijmegen, Netherlands
| | - Li Sun
- Peking University Sixth Hospital/Institute of Mental Health, Beijing, China; National Clinical Research Center for Mental Disorders, Key Laboratory of Mental Health, Ministry of Health, Peking University, Beijing, China
| | - Han Jiang
- College of Special Education, Zhejiang Normal University, Hangzhou, China
| |
Collapse
|
3
|
Semenkov I, Fedosov N, Makarov I, Ossadtchi A. Real-time low latency estimation of brain rhythms with deep neural networks. J Neural Eng 2023; 20:056008. [PMID: 37683653 DOI: 10.1088/1741-2552/acf7f3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2023] [Accepted: 09/08/2023] [Indexed: 09/10/2023]
Abstract
Objective.Neurofeedback and brain-computer interfacing technology open the exciting opportunity for establishing interactive closed-loop real-time communication with the human brain. This requires interpreting brain's rhythmic activity and generating timely feedback to the brain. Lower delay between neuronal events and the appropriate feedback increases the efficacy of such interaction. Novel more efficient approaches capable of tracking brain rhythm's phase and envelope are needed for scenarios that entail instantaneous interaction with the brain circuits.Approach.Isolating narrow-band signals incurs fundamental delays. To some extent they can be compensated using forecasting models. Given the high quality of modern time series forecasting neural networks we explored their utility for low-latency extraction of brain rhythm parameters. We tested five neural networks with conceptually distinct architectures in forecasting synthetic EEG rhythms. The strongest architecture was then trained to simultaneously filter and forecast EEG data. We compared it against the state-of-the-art techniques using synthetic and real data from 25 subjects.Main results.The temporal convolutional network (TCN) remained the strongest forecasting model that achieved in the majority of testing scenarios>90% rhythm's envelope correlation with<10 ms effective delay and<20∘circular standard deviation of phase estimates. It also remained stable enough to noise level perturbations. Trained to filter and predict the TCN outperformed the cFIR, the Kalman filter based state-space estimation technique and remained on par with the larger Conv-TasNet architecture.Significance.Here we have for the first time demonstrated the utility of the neural network approach for low-latency narrow-band filtering of brain activity signals. Our proposed approach coupled with efficient implementation enhances the effectiveness of brain-state dependent paradigms across various applications. Moreover, our framework for forecasting EEG signals holds promise for investigating the predictability of brain activity, providing valuable insights into the fundamental questions surrounding the functional organization and hierarchical information processing properties of the brain.
Collapse
Affiliation(s)
- Ilia Semenkov
- Artificial Intelligence Research Institute (AIRI), Moscow 105064, Russia
- HSE University, Moscow 109028, Russia
| | - Nikita Fedosov
- Artificial Intelligence Research Institute (AIRI), Moscow 105064, Russia
- HSE University, Moscow 109028, Russia
| | - Ilya Makarov
- Artificial Intelligence Research Institute (AIRI), Moscow 105064, Russia
| | - Alexei Ossadtchi
- Artificial Intelligence Research Institute (AIRI), Moscow 105064, Russia
- HSE University, Moscow 109028, Russia
- LLC 'Life Improvement by Future Technologies Center', Moscow, Russia
| |
Collapse
|
4
|
Chikhi S, Matton N, Sanna M, Blanchet S. Mental strategies and resting state EEG: Effect on high alpha amplitude modulation by neurofeedback in healthy young adults. Biol Psychol 2023; 178:108521. [PMID: 36801435 DOI: 10.1016/j.biopsycho.2023.108521] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2022] [Revised: 11/30/2022] [Accepted: 02/15/2023] [Indexed: 02/19/2023]
Abstract
Neurofeedback (NFB) is a brain-computer interface which allows individuals to modulate their brain activity. Despite the self-regulatory nature of NFB, the effectiveness of strategies used during NFB training has been little investigated. In a single session of NFB training (6*3 min training blocks) with healthy young participants, we experimentally tested if providing a list of mental strategies (list group, N = 46), compared with a group receiving no strategies (no list group, N = 39), affected participants' neuromodulation ability of high alpha (10-12 Hz) amplitude. We additionally asked participants to verbally report the mental strategies used to enhance high alpha amplitude. The verbatim was then classified in pre-established categories in order to examine the effect of type of mental strategy on high alpha amplitude. First, we found that giving a list to the participants did not promote the ability to neuromodulate high alpha activity. However, our analysis of the specific strategies reported by learners during training blocks revealed that cognitive effort and recalling memories were associated with higher high alpha amplitude. Furthermore, the resting amplitude of trained high alpha frequency predicted an amplitude increase during training, a factor that may optimize inclusion in NFB protocols. The present results also corroborate the interrelation with other frequency bands during NFB training. Although these findings are based on a single NFB session, our study represents a further step towards developing effective protocols for high alpha neuromodulation by NFB.
Collapse
Affiliation(s)
- Samy Chikhi
- Université Paris Cité, Laboratoire Mémoire, Cerveau et Cognition, F-92100 Boulogne-Billancourt, France
| | - Nadine Matton
- CLLE, Université de Toulouse, CNRS (UMR 5263), Toulouse, France; ENAC, École Nationale d'Aviation Civile, Université de Toulouse, France
| | - Marie Sanna
- Université Paris Cité, Laboratoire Mémoire, Cerveau et Cognition, F-92100 Boulogne-Billancourt, France
| | - Sophie Blanchet
- Université Paris Cité, Laboratoire Mémoire, Cerveau et Cognition, F-92100 Boulogne-Billancourt, France.
| |
Collapse
|
5
|
Kvamme TL, Ros T, Overgaard M. Can neurofeedback provide evidence of direct brain-behavior causality? Neuroimage 2022; 258:119400. [PMID: 35728786 DOI: 10.1016/j.neuroimage.2022.119400] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Revised: 06/13/2022] [Accepted: 06/17/2022] [Indexed: 01/01/2023] Open
Abstract
Neurofeedback is a procedure that measures brain activity in real-time and presents it as feedback to an individual, thus allowing them to self-regulate brain activity with effects on cognitive processes inferred from behavior. One common argument is that neurofeedback studies can reveal how the measured brain activity causes a particular cognitive process. The causal claim is often made regarding the measured brain activity being manipulated as an independent variable, similar to brain stimulation studies. However, this causal inference is vulnerable to the argument that other upstream brain activities change concurrently and cause changes in the brain activity from which feedback is derived. In this paper, we outline the inference that neurofeedback may causally affect cognition by indirect means. We further argue that researchers should remain open to the idea that the trained brain activity could be part of a "causal network" that collectively affects cognition rather than being necessarily causally primary. This particular inference may provide a better translation of evidence from neurofeedback studies to the rest of neuroscience. We argue that the recent advent of multivariate pattern analysis, when combined with implicit neurofeedback, currently comprises the strongest case for causality. Our perspective is that although the burden of inferring direct causality is difficult, it may be triangulated using a collection of various methods in neuroscience. Finally, we argue that the neurofeedback methodology provides unique advantages compared to other methods for revealing changes in the brain and cognitive processes but that researchers should remain mindful of indirect causal effects.
Collapse
Affiliation(s)
- Timo L Kvamme
- Cognitive Neuroscience Research Unit, CFIN/MINDLab, Aarhus University, Universitetsbyen 3, Aarhus, Denmark; Centre for Alcohol and Drug Research (CRF), Aarhus University, Aarhus, Denmark.
| | - Tomas Ros
- Departments of Neuroscience and Psychiatry, University of Geneva, Campus Biotech, Geneva, Switzerland
| | - Morten Overgaard
- Cognitive Neuroscience Research Unit, CFIN/MINDLab, Aarhus University, Universitetsbyen 3, Aarhus, Denmark
| |
Collapse
|
6
|
Zhou Q, Cheng R, Yao L, Ye X, Xu K. Neurofeedback Training of Alpha Relative Power Improves the Performance of Motor Imagery Brain-Computer Interface. Front Hum Neurosci 2022; 16:831995. [PMID: 35463935 PMCID: PMC9026187 DOI: 10.3389/fnhum.2022.831995] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2021] [Accepted: 03/16/2022] [Indexed: 01/03/2023] Open
Abstract
Significant variation in performance in motor imagery (MI) tasks impedes their wide adoption for brain-computer interface (BCI) applications. Previous researchers have found that resting-state alpha-band power is positively correlated with MI-BCI performance. In this study, we designed a neurofeedback training (NFT) protocol based on the up-regulation of the alpha band relative power (RP) to investigate its effect on MI-BCI performance. The principal finding of this study is that alpha NFT could successfully help subjects increase alpha-rhythm power and improve their MI-BCI performance. An individual difference was also found in this study in that subjects who increased alpha power more had a better performance improvement. Additionally, the functional connectivity (FC) of the frontal-parietal (FP) network was found to be enhanced after alpha NFT. However, the enhancement failed to reach a significant level after multiple comparisons correction. These findings contribute to a better understanding of the neurophysiological mechanism of cognitive control through alpha regulation.
Collapse
Affiliation(s)
- Qing Zhou
- Qiushi Academy for Advanced Studies (QAAS), Zhejiang University, Hangzhou, China
- Zhejiang Lab, Hangzhou, China
- Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Key Laboratory of Biomedical Engineering of Education Ministry, Zhejiang University, Hangzhou, China
| | - Ruidong Cheng
- Center for Rehabilitation Medicine, Rehabilitation and Sports Medicine Research Institute of Zhejiang Province, Department of Rehabilitation Medicine, Zhejiang Provincial People’s Hospital (Affiliated People’s Hospital, Hangzhou Medical College), Hangzhou, China
| | - Lin Yao
- MOE Frontiers Science Center for Brain and Brain-Machine Integration, Zhejiang University, Hangzhou, China
- Department of Neurobiology, Affiliated Mental Health Center and Hangzhou Seventh People’s Hospital, Zhejiang University School of Medicine, Hangzhou, China
- The College of Computer Science, Zhejiang University, Hangzhou, China
| | - Xiangming Ye
- Center for Rehabilitation Medicine, Rehabilitation and Sports Medicine Research Institute of Zhejiang Province, Department of Rehabilitation Medicine, Zhejiang Provincial People’s Hospital (Affiliated People’s Hospital, Hangzhou Medical College), Hangzhou, China
- Xiangming Ye,
| | - Kedi Xu
- Qiushi Academy for Advanced Studies (QAAS), Zhejiang University, Hangzhou, China
- Zhejiang Lab, Hangzhou, China
- Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Key Laboratory of Biomedical Engineering of Education Ministry, Zhejiang University, Hangzhou, China
- MOE Frontiers Science Center for Brain and Brain-Machine Integration, Zhejiang University, Hangzhou, China
- *Correspondence: Kedi Xu,
| |
Collapse
|
7
|
Asai T, Hamamoto T, Kashihara S, Imamizu H. Real-Time Detection and Feedback of Canonical Electroencephalogram Microstates: Validating a Neurofeedback System as a Function of Delay. Front Syst Neurosci 2022; 16:786200. [PMID: 35283737 PMCID: PMC8913511 DOI: 10.3389/fnsys.2022.786200] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Accepted: 02/04/2022] [Indexed: 11/13/2022] Open
Abstract
Recent neurotechnology has developed various methods for neurofeedback (NF), in which participants observe their own neural activity to be regulated in an ideal direction. EEG-microstates (EEGms) are spatially featured states that can be regulated through NF training, given that they have recently been indicated as biomarkers for some disorders. The current study was conducted to develop an EEG-NF system for detecting “canonical 4 EEGms” in real time. There are four representative EEG states, regardless of the number of channels, preprocessing procedures, or participants. Accordingly, our 10 Hz NF system was implemented to detect them (msA, B, C, and D) and audio-visually inform participants of its detection. To validate the real-time effect of this system on participants’ performance, the NF was intentionally delayed for participants to prevent their cognitive control in learning. Our results suggest that the feedback effect was observed only under the no-delay condition. The number of Hits increased significantly from the baseline period and increased from the 1- or 20-s delay conditions. In addition, when the Hits were compared among the msABCD, each cognitive or perceptual function could be characterized, though the correspondence between each microstate and psychological ability might not be that simple. For example, msD should be generally task-positive and less affected by the inserted delay, whereas msC is more delay-sensitive. In this study, we developed and validated a new EEGms-NF system as a function of delay. Although the participants were naive to the inserted delay, the real-time NF successfully increased their Hit performance, even within a single-day experiment, although target specificity remains unclear. Future research should examine long-term training effects using this NF system.
Collapse
Affiliation(s)
- Tomohisa Asai
- Cognitive Mechanisms Laboratories, Advanced Telecommunications Research Institute International (ATR), Kyoto, Japan
- *Correspondence: Tomohisa Asai,
| | - Takamasa Hamamoto
- Cognitive Mechanisms Laboratories, Advanced Telecommunications Research Institute International (ATR), Kyoto, Japan
- Graduate School of Frontier Biosciences, Osaka University, Osaka, Japan
| | - Shiho Kashihara
- Cognitive Mechanisms Laboratories, Advanced Telecommunications Research Institute International (ATR), Kyoto, Japan
| | - Hiroshi Imamizu
- Cognitive Mechanisms Laboratories, Advanced Telecommunications Research Institute International (ATR), Kyoto, Japan
- Department of Psychology, Graduate School of Humanities and Sociology, The University of Tokyo, Tokyo, Japan
| |
Collapse
|
8
|
Improving Functional Connectivity in Developmental Dyslexia through Combined Neurofeedback and Visual Training. Symmetry (Basel) 2022. [DOI: 10.3390/sym14020369] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
This study examined the effects of combined neurofeedback (NF) and visual training (VT) on children with developmental dyslexia (DD). Although NF is the first noninvasive approach to support neurological disorders, the mechanisms of its effects on the brain functional connectivity are still unclear. A key question is whether the functional connectivities of the EEG frequency networks change after the combined NF–VT training of DD children (postD). NF sessions of voluntary α/θ rhythm control were applied in a low-spatial-frequency (LSF) illusion contrast discrimination, which provides feedback with visual cues to improve the brain signals and cognitive abilities in DD children. The measures of connectivity, which are defined by small-world propensity, were sensitive to the properties of the brain electrical oscillations in the quantitative EEG-NF training. In the high-contrast LSF illusion, the z-NF reduced the α/θ scores in the frontal areas, and in the right ventral temporal, occipital–temporal, and middle occipital areas in the postD (vs. the preD) because of their suppression in the local hub θ-network and the altered global characteristics of the functional θ-frequency network. In the low-contrast condition, the z-NF stimulated increases in the α/θ scores, which induced hubs in the left-side α-frequency network of the postD, and changes in the global characteristics of the functional α-frequency network. Because of the anterior, superior, and middle temporal deficits affecting the ventral and occipital–temporal pathways, the z-NF–VT compensated for the more ventral brain regions, mainly in the left hemispheres of the postD group in the low-contrast LSF illusion. Compared to pretraining, the NF–VT increased the segregation of the α, β (low-contrast), and θ networks (high-contrast), as well as the γ2-network integration (both contrasts) after the termination of the training of the children with developmental dyslexia. The remediation compensated more for the dorsal (prefrontal, premotor, occipital–parietal connectivities) dysfunction of the θ network in the developmental dyslexia in the high-contrast LSF illusion. Our findings provide neurobehavioral evidence for the exquisite brain functional plasticity and direct effect of NF–VT on cognitive disabilities in DD children.
Collapse
|
9
|
Peterson V, Nieto N, Wyser D, Lambercy O, Gassert R, Milone DH, Spies RD. Transfer Learning based on Optimal Transport for Motor Imagery Brain-Computer Interfaces. IEEE Trans Biomed Eng 2021; 69:807-817. [PMID: 34406935 DOI: 10.1109/tbme.2021.3105912] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
OBJECTIVE This paper tackles the cross-sessions variability of electroencephalography-based brain-computer interfaces (BCIs) in order to avoid the lengthy recalibration step of the decoding method before every use. METHODS We develop a new approach of domain adaptation based on optimal transport to tackle brain signal variability between sessions of motor imagery BCIs. We propose a backward method where, unlike the original formulation, the data from a new session are transported to a calibration session, and thereby avoiding model retraining. Several domain adaptation approaches are evaluated and compared. We simulated two possible online scenarios: i) block-wise adaptation and ii) sample-wise adaptation. In this study, we collect a dataset of 10 subjects performing a hand motor imagery task in 2 sessions. A publicly available dataset is also used. RESULTS For the first scenario, results indicate that classifier retraining can be avoided by means of our backward formulation yielding to equivalent classification performance as compared to retraining solutions. In the second scenario, classification performance rises up to 90.23\% overall accuracy when the label of the indicated mental task is used to learn the transport. Adaptive time is between 10 and 80 times faster than the other methods. CONCLUSIONS The proposed method is able to mitigate the cross-session variability in motor imagery BCIs. SIGNIFICANCE The backward formulation is an efficient retraining-free approach built to avoid lengthy calibration times. Thus, the BCI can be actively used after just a few minutes of setup. This is important for practical applications such as BCI-based motor rehabilitation.
Collapse
|