Two-stage regression of high-density scalp electroencephalograms visualizes force regulation signaling during muscle contraction

EEG decoding with spatiotemporal convolutional neural network for visualization and closed-loop control of sensorimotor activities: A simultaneous EEG-fMRI study

Closed-loop neurofeedback training utilizes neural signals such as scalp electroencephalograms (EEG) to manipulate specific neural activities and the associated behavioral performance. A spatiotemporal filter for high-density whole-head scalp EEG using a convolutional neural network can overcome the ambiguity of the signaling source because each EEG signal includes information on the remote regions. We simultaneously acquired EEG and functional magnetic resonance images in humans during the brain-computer interface (BCI) based neurofeedback training and compared the reconstructed and modeled hemodynamic responses of the sensorimotor network. Filters constructed with a convolutional neural network captured activities in the targeted network with spatial precision and specificity superior to those of the EEG signals preprocessed with standard pipelines used in BCI-based neurofeedback paradigms. The middle layers of the trained model were examined to characterize the neuronal oscillatory features that contributed to the reconstruction. Analysis of the layers for spatial convolution revealed the contribution of distributed cortical circuitries to reconstruction, including the frontoparietal and sensorimotor areas, and those of temporal convolution layers that successfully reconstructed the hemodynamic response function. Employing a spatiotemporal filter and leveraging the electrophysiological signatures of the sensorimotor excitability identified in our middle layer analysis would contribute to the development of a further effective neurofeedback intervention.

Spatially bivariate EEG-neurofeedback can manipulate interhemispheric inhibition

Human behavior requires inter-regional crosstalk to employ the sensorimotor processes in the brain. Although external neuromodulation techniques have been used to manipulate interhemispheric sensorimotor activity, a central controversy concerns whether this activity can be volitionally controlled. Experimental tools lack the power to up- or down-regulate the state of the targeted hemisphere over a large dynamic range and, therefore, cannot evaluate the possible volitional control of the activity. We addressed this difficulty by using the recently developed method of spatially bivariate electroencephalography (EEG)-neurofeedback to systematically enable the participants to modulate their bilateral sensorimotor activities. Here, we report that participants learn to up- and down-regulate the ipsilateral excitability to the imagined hand while maintaining constant contralateral excitability; this modulates the magnitude of interhemispheric inhibition (IHI) assessed by the paired-pulse transcranial magnetic stimulation (TMS) paradigm. Further physiological analyses revealed that the manipulation capability of IHI magnitude reflected interhemispheric connectivity in EEG and TMS, which was accompanied by intrinsic bilateral cortical oscillatory activities. Our results show an interesting approach for neuromodulation, which might identify new treatment opportunities, e.g., in patients suffering from a stroke.

Deep learning multimodal fNIRS and EEG signals for bimanual grip force decoding

Objective.Non-invasive brain-machine interfaces (BMIs) offer an alternative, safe and accessible way to interact with the environment. To enable meaningful and stable physical interactions, BMIs need to decode forces. Although previously addressed in the unimanual case, controlling forces from both hands would enable BMI-users to perform a greater range of interactions. We here investigate the decoding of hand-specific forces.Approach.We maximise cortical information by using electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) and developing a deep-learning architecture with attention and residual layers (cnnatt) to improve their fusion. Our task required participants to generate hand-specific force profiles on which we trained and tested our deep-learning and linear decoders.Main results.The use of EEG and fNIRS improved the decoding of bimanual force and the deep-learning models outperformed the linear model. In both cases, the greatest gain in performance was due to the detection of force generation. In particular, the detection of forces was hand-specific and better for the right dominant hand andcnnattwas better at fusing EEG and fNIRS. Consequently, the study ofcnnattrevealed that forces from each hand were differently encoded at the cortical level.Cnnattalso revealed traces of the cortical activity being modulated by the level of force which was not previously found using linear models.Significance.Our results can be applied to avoid hand-cross talk during hand force decoding to improve the robustness of BMI robotic devices. In particular, we improve the fusion of EEG and fNIRS signals and offer hand-specific interpretability of the encoded forces which are valuable during motor rehabilitation assessment.

Deep Neural Regression Prediction of Motor Imagery Skills Using EEG Functional Connectivity Indicators

Motor imaging (MI) induces recovery and neuroplasticity in neurophysical regulation. However, a non-negligible portion of users presents insufficient coordination skills of sensorimotor cortex control. Assessments of the relationship between wakefulness and tasks states are conducted to foster neurophysiological and mechanistic interpretation in MI-related applications. Thus, to understand the organization of information processing, measures of functional connectivity are used. Also, models of neural network regression prediction are becoming popular, These intend to reduce the need for extracting features manually. However, predicting MI practicing's neurophysiological inefficiency raises several problems, like enhancing network regression performance because of the overfitting risk. Here, to increase the prediction performance, we develop a deep network regression model that includes three procedures: leave-one-out cross-validation combined with Monte Carlo dropout layers, subject clustering of MI inefficiency, and transfer learning between neighboring runs. Validation is performed using functional connectivity predictors extracted from two electroencephalographic databases acquired in conditions close to real MI applications (150 users), resulting in a high prediction of pretraining desynchronization and initial training synchronization with adequate physiological interpretability.

Scalp electroencephalograms over ipsilateral sensorimotor cortex reflect contraction patterns of unilateral finger muscles

A variety of neural substrates are implicated in the initiation, coordination, and stabilization of voluntary movements underpinned by adaptive contraction and relaxation of agonist and antagonist muscles. To achieve such flexible and purposeful control of the human body, brain systems exhibit extensive modulation during the transition from resting state to motor execution and to maintain proper joint impedance. However, the neural structures contributing to such sensorimotor control under unconstrained and naturalistic conditions are not fully characterized. To elucidate which brain regions are implicated in generating and coordinating voluntary movements, we employed a physiologically inspired, two-stage method to decode relaxation and three patterns of contraction in unilateral finger muscles (i.e., extension, flexion, and co-contraction) from high-density scalp electroencephalograms (EEG). The decoder consisted of two parts employed in series. The first discriminated between relaxation and contraction. If the EEG data were discriminated as contraction, the second stage then discriminated among the three contraction patterns. Despite the difficulty in dissociating detailed contraction patterns of muscles within a limb from scalp EEG signals, the decoder performance was higher than chance-level by 2-fold in the four-class classification. Moreover, weighted features in the trained decoders revealed EEG features differentially contributing to decoding performance. During the first stage, consistent with previous reports, weighted features were localized around sensorimotor cortex (SM1) contralateral to the activated fingers, while those during the second stage were localized around ipsilateral SM1. The loci of these weighted features suggested that the coordination of unilateral finger muscles induced different signaling patterns in ipsilateral SM1 contributing to motor control. Weighted EEG features enabled a deeper understanding of human sensorimotor processing as well as of a more naturalistic control of brain-computer interfaces.

Neurofeedback of scalp bi-hemispheric EEG sensorimotor rhythm guides hemispheric activation of sensorimotor cortex in the targeted hemisphere

Oscillatory electroencephalographic (EEG) activity is associated with the excitability of cortical regions. Visual feedback of EEG-oscillations may promote sensorimotor cortical activation, but its spatial specificity is not truly guaranteed due to signal interaction among interhemispheric brain regions. Guiding spatially specific activation is important for facilitating neural rehabilitation processes. Here, we tested whether users could explicitly guide sensorimotor cortical activity to the contralateral or ipsilateral hemisphere using a spatially bivariate EEG-based neurofeedback that monitors bi-hemispheric sensorimotor cortical activities for healthy participants. Two different motor imageries (shoulder and hand MIs) were selected to see how differences in intrinsic corticomuscular projection patterns might influence activity lateralization. We showed sensorimotor cortical activities during shoulder, but not hand MI, can be brought under ipsilateral control with guided EEG-based neurofeedback. These results are compatible with neuroanatomy; shoulder muscles are innervated bihemispherically, whereas hand muscles are mostly innervated contralaterally. We demonstrate the neuroanatomically-inspired approach enables us to investigate potent neural remodeling functions that underlie EEG-based neurofeedback via a BCI.