Electrocorticographic spectral changes associated with ipsilateral individual finger and whole hand movement

Unleashing the potential of fNIRS with machine learning: classification of fine anatomical movements to empower future brain-computer interface

In this study, we explore the potential of using functional near-infrared spectroscopy (fNIRS) signals in conjunction with modern machine-learning techniques to classify specific anatomical movements to increase the number of control commands for a possible fNIRS-based brain-computer interface (BCI) applications. The study focuses on novel individual finger-tapping, a well-known task in fNIRS and fMRI studies, but limited to left/right or few fingers. Twenty-four right-handed participants performed the individual finger-tapping task. Data were recorded by using sixteen sources and detectors placed over the motor cortex according to the 10-10 international system. The event's average oxygenated Δ HbO and deoxygenated Δ HbR hemoglobin data were utilized as features to assess the performance of diverse machine learning (ML) models in a challenging multi-class classification setting. These methods include LDA, QDA, MNLR, XGBoost, and RF. A new DL-based model named "Hemo-Net" has been proposed which consists of multiple parallel convolution layers with different filters to extract the features. This paper aims to explore the efficacy of using fNRIS along with ML/DL methods in a multi-class classification task. Complex models like RF, XGBoost, and Hemo-Net produce relatively higher test set accuracy when compared to LDA, MNLR, and QDA. Hemo-Net has depicted a superior performance achieving the highest test set accuracy of 76%, however, in this work, we do not aim at improving the accuracies of models rather we are interested in exploring if fNIRS has the neural signatures to help modern ML/DL methods in multi-class classification which can lead to applications like brain-computer interfaces. Multi-class classification of fine anatomical movements, such as individual finger movements, is difficult to classify with fNIRS data. Traditional ML models like MNLR and LDA show inferior performance compared to the ensemble-based methods of RF and XGBoost. DL-based method Hemo-Net outperforms all methods evaluated in this study and demonstrates a promising future for fNIRS-based BCI applications.

Spatiotemporal dynamics of human high gamma discriminate naturalistic behavioral states

In analyzing the neural correlates of naturalistic and unstructured behaviors, features of neural activity that are ignored in a trial-based experimental paradigm can be more fully studied and investigated. Here, we analyze neural activity from two patients using electrocorticography (ECoG) and stereo-electroencephalography (sEEG) recordings, and reveal that multiple neural signal characteristics exist that discriminate between unstructured and naturalistic behavioral states such as “engaging in dialogue” and “using electronics”. Using the high gamma amplitude as an estimate of neuronal firing rate, we demonstrate that behavioral states in a naturalistic setting are discriminable based on long-term mean shifts, variance shifts, and differences in the specific neural activity’s covariance structure. Both the rapid and slow changes in high gamma band activity separate unstructured behavioral states. We also use Gaussian process factor analysis (GPFA) to show the existence of salient spatiotemporal features with variable smoothness in time. Further, we demonstrate that both temporally smooth and stochastic spatiotemporal activity can be used to differentiate unstructured behavioral states. This is the first attempt to elucidate how different neural signal features contain information about behavioral states collected outside the conventional experimental paradigm.

Continuous Wavelet Transform for Decoding Finger Movements From Single-Channel EEG

Human body movements can be reflected in brain signals and collected noninvasively with electroencephalography (EEG). Motor-related signals include sensory motor rhythms (also known as the Mu wave) in the upper-alpha band of 8-13 Hz and slow cortical potentials (SCPs) in the low frequency range of 0.1-5 Hz. This study compares the two signals for decoding finger movements. Human subjects were asked to individually lift each of the five digits of their right hand, at the rate of one every 10 s. EEG was recorded using a bipolar montage between ipsilateral and contralateral motor cortices. Electromyograms were obtained for identifying movement onsets. Linear discriminant analysis (LDA) generated significant performance with SCPs but not with Mu. Meanwhile, continuous wavelet transform (CWT) was applied to SCPs or Mu to create a spectrogram for each finger, showing distinctive amplitude and phase patterns. A dprime-based weighting algorithm was used to extract time-frequency features. With a template-matching paradigm, both SCP and Mu spectrograms yielded significant classification accuracies for multiple subjects, with the highest being >50% (chance = 20%). Interestingly, the index finger was better distinguished with Mu for most of the subjects, whereas the ring finger was better distinguished with SCPs. The CWT algorithm outperformed LDA by better decoding the thumb. This study suggests that the time-frequency characteristics of a single-channel EEG, when phase is preserved, contain critical information on finger movements. SCPs and Mu seem to work in an independent but complementary manner.

Decoding individual finger movements from one hand using human EEG signals

Brain computer interface (BCI) is an assistive technology, which decodes neurophysiological signals generated by the human brain and translates them into control signals to control external devices, e.g., wheelchairs. One problem challenging noninvasive BCI technologies is the limited control dimensions from decoding movements of, mainly, large body parts, e.g., upper and lower limbs. It has been reported that complicated dexterous functions, i.e., finger movements, can be decoded in electrocorticography (ECoG) signals, while it remains unclear whether noninvasive electroencephalography (EEG) signals also have sufficient information to decode the same type of movements. Phenomena of broadband power increase and low-frequency-band power decrease were observed in EEG in the present study, when EEG power spectra were decomposed by a principal component analysis (PCA). These movement-related spectral structures and their changes caused by finger movements in EEG are consistent with observations in previous ECoG study, as well as the results from ECoG data in the present study. The average decoding accuracy of 77.11% over all subjects was obtained in classifying each pair of fingers from one hand using movement-related spectral changes as features to be decoded using a support vector machine (SVM) classifier. The average decoding accuracy in three epilepsy patients using ECoG data was 91.28% with the similarly obtained features and same classifier. Both decoding accuracies of EEG and ECoG are significantly higher than the empirical guessing level (51.26%) in all subjects (p<0.05). The present study suggests the similar movement-related spectral changes in EEG as in ECoG, and demonstrates the feasibility of discriminating finger movements from one hand using EEG. These findings are promising to facilitate the development of BCIs with rich control signals using noninvasive technologies.

Fast presurgical functional mapping using task-related intracranial high gamma activity

OBJECT

Electrocorticography (ECoG) is a powerful tool for presurgical functional mapping. Power increase in the high gamma band has been observed from ECoG electrodes on the surface of the sensory motor cortex during the execution of body movements. In this study the authors aim to validate the clinical usage of high gamma activity in presurgical mapping by comparing ECoG mapping with traditional direct electrical cortical stimulation (ECS) and functional MRI (fMRI) mapping.

METHODS

Seventeen patients with epilepsy participated in an ECoG motor mapping experiment. The patients executed a 5-minute hand/tongue movement task while the ECoG signal was recorded. All 17 patients also underwent extraoperative ECS mapping to localize the motor cortex. Eight patients also participated in a presurgical fMRI study. The high gamma activity on ECoG was modeled using the general linear model (GLM), and the regions showing significant gamma power increase during the task condition compared with the rest condition were localized. The maps derived from GLM-based ECoG mapping, ECS, and fMRI were then compared.

RESULTS

High gamma activity in the motor cortex can be reliably modulated by motor tasks. Localization of the motor regions achieved with GLM-based ECoG mapping was consistent with the localization determined by ECS. The maps also appeared to be highly localized compared with the fMRI activations. Using the ECS findings as the reference, GLM-based ECoG mapping showed a significantly higher sensitivity than fMRI (66.7% for ECoG, 52.6% for fMRI, p<0.05), while the specificity was high for both techniques (>97%). If the current-spreading effect in ECS is accounted for, ECoG mapping may produce maps almost identical to those produced by ECS mapping (100% sensitivity and 99.5% specificity).

CONCLUSIONS

General linear model-based ECoG mapping showed a superior performance compared to traditional ECS and fMRI mapping in terms of efficiency and accuracy. Using this method, motor functions can be reliably mapped in less than 5 minutes.

ECoG based cortical function mapping using general linear model

Electrocorticography (ECoG) is an emerging tool to map brain functions in the context of neurosurgical intervention. Previous mapping methods based on the event related power spectrum are prone to noise. To improve the robustness of cortical function mapping, general linear model (GLM), which has been widely used in the analysis of functional magnetic resonance imaging (fMRI) data, is applied to bandpass filtered ECoG signals from each electrode. For a specific task, electrodes with best fitting parameters of the signal are identified, and the statistical significance of the fitting is mapped on the standard 3D brain model to provide a personalized map of sensorimotor functions. With the analysis of four patients' data, the proposed approach yields consistent results with those obtained by electrical cortical stimulation (ECS), while showing promising performance against noise.

Using ipsilateral motor signals in the unaffected cerebral hemisphere as a signal platform for brain-computer interfaces in hemiplegic stroke survivors

Brain-computer interface (BCI) systems have emerged as a method to restore function and enhance communication in motor impaired patients. To date, this has been applied primarily to patients who have a compromised motor outflow due to spinal cord dysfunction, but an intact and functioning cerebral cortex. The cortical physiology associated with movement of the contralateral limb has typically been the signal substrate that has been used as a control signal. While this is an ideal control platform in patients with an intact motor cortex, these signals are lost after a hemispheric stroke. Thus, a different control signal is needed that could provide control capability for a patient with a hemiparetic limb. Previous studies have shown that there is a distinct cortical physiology associated with ipsilateral, or same-sided, limb movements. Thus far, it was unknown whether stroke survivors could intentionally and effectively modulate this ipsilateral motor activity from their unaffected hemisphere. Therefore, this study seeks to evaluate whether stroke survivors could effectively utilize ipsilateral motor activity from their unaffected hemisphere to achieve this BCI control. To investigate this possibility, electroencephalographic (EEG) signals were recorded from four chronic hemispheric stroke patients as they performed (or attempted to perform) real and imagined hand tasks using either their affected or unaffected hand. Following performance of the screening task, the ability of patients to utilize a BCI system was investigated during on-line control of a one-dimensional control task. Significant ipsilateral motor signals (associated with movement intentions of the affected hand) in the unaffected hemisphere, which were found to be distinct from rest and contralateral signals, were identified and subsequently used for a simple online BCI control task. We demonstrate here for the first time that EEG signals from the unaffected hemisphere, associated with overt and imagined movements of the affected hand, can enable stroke survivors to control a one-dimensional computer cursor rapidly and accurately. This ipsilateral motor activity enabled users to achieve final target accuracies between 68% and 91% within 15 min. These findings suggest that ipsilateral motor activity from the unaffected hemisphere in stroke survivors could provide a physiological substrate for BCI operation that can be further developed as a long-term assistive device or potentially provide a novel tool for rehabilitation.

Transient and state modulation of beta power in human subthalamic nucleus during speech production and finger movement

Signs of Parkinson's disease (PD) are augmented by speech and repetitive motor tasks. The neurophysiological basis for this phenomenon is unknown, but may involve augmentation of β (13-30 Hz) oscillations within the subthalamic nucleus (STN). We hypothesized that speech and motor tasks increase β power in STN and propose a mechanism for clinical observations of worsening motor state during such behaviors. Subjects undergoing deep brain stimulation (DBS) surgery performed tasks while STN local field potential (LFP) data were collected. Power in the β frequency range was analyzed across the entire recording to observe slow shifts related to block design and during time epochs synchronized to behavior to evaluate immediate fluctuations related to task execution. Bilaterally symmetric β event related desynchronization was observed in analysis time-locked to subject motor and speech tasks. We also observed slow shifts of β power associated with blocks of tasks. Repetitive combined speech and motor, and isolated motor blocks were associated with the highest bilateral β power state. Overt speech alone and imagined speech were associated with a low bilateral β power state. Thus, changing behavioral tasks is associated with bilateral switching of β power states. This offers a potential neurophysiologic correlate of worsened PD motor signs experienced during clinical examination with provocative tasks: switching into a high β power state may be responsible for worsening motor states in PD patients when performing unilateral repetitive motor tasks and combined speech and motor tasks. Beta state changes could be chronically measured and potentially used to control closed loop neuromodulatory devices in the future.

Cortical gamma-oscillations modulated by auditory-motor tasks-intracranial recording in patients with epilepsy

Human activities often involve hand-motor responses following external auditory-verbal commands. It has been believed that hand movements are predominantly driven by the contralateral primary sensorimotor cortex, whereas auditory-verbal information is processed in both superior temporal gyri. It remains unknown whether cortical activation in the superior temporal gyrus during an auditory-motor task is affected by laterality of hand-motor responses. Here, event-related γ-oscillations were intracranially recorded as quantitative measures of cortical activation; we determined how cortical structures were activated by auditory-cued movement using each hand in 15 patients with focal epilepsy. Auditory-verbal stimuli elicited augmentation of γ-oscillations in a posterior portion of the superior temporal gyrus, whereas hand-motor responses elicited γ-augmentation in the pre- and postcentral gyri. The magnitudes of such γ-augmentation in the superior temporal, precentral, and postcentral gyri were significantly larger when the hand contralateral to the recorded hemisphere was required to be used for motor responses, compared with when the ipsilateral hand was. The superior temporal gyrus in each hemisphere might play a greater pivotal role when the contralateral hand needs to be used for motor responses, compared with when the ipsilateral hand does.

Decoding spoken words using local field potentials recorded from the cortical surface

Pathological conditions such as amyotrophic lateral sclerosis or damage to the brainstem can leave patients severely paralyzed but fully aware, in a condition known as 'locked-in syndrome'. Communication in this state is often reduced to selecting individual letters or words by arduous residual movements. More intuitive and rapid communication may be restored by directly interfacing with language areas of the cerebral cortex. We used a grid of closely spaced, nonpenetrating micro-electrodes to record local field potentials (LFPs) from the surface of face motor cortex and Wernicke's area. From these LFPs we were successful in classifying a small set of words on a trial-by-trial basis at levels well above chance. We found that the pattern of electrodes with the highest accuracy changed for each word, which supports the idea that closely spaced micro-electrodes are capable of capturing neural signals from independent neural processing assemblies. These results further support using cortical surface potentials (electrocorticography) in brain-computer interfaces. These results also show that LFPs recorded from the cortical surface (micro-electrocorticography) of language areas can be used to classify speech-related cortical rhythms and potentially restore communication to locked-in patients.

Electrocorticographic amplitude predicts finger positions during slow grasping motions of the hand

Four human subjects undergoing subdural electrocorticography for epilepsy surgery engaged in a range of finger and hand movements. We observed that the amplitudes of the low-pass filtered electrocorticogram (ECoG), also known as the local motor potential (LMP), over specific peri-Rolandic electrodes were correlated (p < 0.001) with the position of individual fingers as the subjects engaged in slow and deliberate grasping motions. A generalized linear model (GLM) of the LMP amplitudes from those electrodes yielded predictions for positions of the fingers that had a strong congruence with the actual finger positions (correlation coefficient, r; median = 0.51, maximum = 0.91), during displacements of up to 10 cm at the fingertips. For all the subjects, decoding filters trained on data from any given session were remarkably robust in their prediction performance across multiple sessions and days, and were invariant with respect to changes in wrist angle, elbow flexion and hand placement across these sessions (median r = 0.52, maximum r = 0.86). Furthermore, a reasonable prediction accuracy for grasp aperture was achievable with as few as three electrodes in all subjects (median r = 0.49; maximum r = 0.90). These results provide further evidence for the feasibility of robust and practical ECoG-based control of finger movements in upper extremity prosthetics.

Classification of spoken words using surface local field potentials

Cortical surface potentials recorded by electrocorticography (ECoG) have enabled robust motor classification algorithms in large part because of the close proximity of the electrodes to the cortical surface. However, standard clinical ECoG electrodes are large in both diameter and spacing relative to the underlying cortical column architecture in which groups of neurons process similar types of stimuli. The potential for surface micro-electrodes closely spaced together to provide even higher fidelity in recording surface field potentials has been a topic of recent interest in the neural prosthetic community. This study describes the classification of spoken words from surface local field potentials (LFPs) recorded using grids of subdural, nonpenetrating high impedance micro-electrodes. Data recorded from these micro-ECoG electrodes supported accurate and rapid classification. Furthermore, electrodes spaced millimeters apart demonstrated varying classification characteristics, suggesting that cortical surface LFPs may be recorded with high temporal and spatial resolution to enable even more robust algorithms for motor classification.

Decoding flexion of individual fingers using electrocorticographic signals in humans

Brain signals can provide the basis for a non-muscular communication and control system, a brain-computer interface (BCI), for people with motor disabilities. A common approach to creating BCI devices is to decode kinematic parameters of movements using signals recorded by intracortical microelectrodes. Recent studies have shown that kinematic parameters of hand movements can also be accurately decoded from signals recorded by electrodes placed on the surface of the brain (electrocorticography (ECoG)). In the present study, we extend these results by demonstrating that it is also possible to decode the time course of the flexion of individual fingers using ECoG signals in humans, and by showing that these flexion time courses are highly specific to the moving finger. These results provide additional support for the hypothesis that ECoG could be the basis for powerful clinically practical BCI systems, and also indicate that ECoG is useful for studying cortical dynamics related to motor function.