Decoding of object categories from brain signals using cross frequency coupling methods

Cross-Frequency Couplings Reveal Mice Visual Cortex Selectivity to Grating Orientations

INTRODUCTION

Oriented grating is usually employed in visual science experiments as a prominent property of neurons in the visual cortices. Previous studies have shown that the study of mouse vision can make a significant contribution to the field of neuroscience research, and also the local field potential (LFP) analysis could contain more information and give us a better view of brain function.

METHODS

In this research, cross-frequency coupling is employed to assess the grating orientation perception in V1 and lateromedial (LM) of 10 mice. The experimental data were collected using chronically implanted multielectrode arrays, involving area V1 recording of five mice and area LM recording of five mice separately, performing a passive visual task. Two criteria known as phase-amplitude coupling (PAC) and amplitude-amplitude coupling (AAC) were exploited to analyze the characteristics of cross-frequency coupling of LFP signals in the experiment consisting of first-order and second-order drifting sinusoidal grating stimuli with different orientations.

RESULTS

It was found that in area LM the correlation between phase of lower than 8 Hz band signal and amplitude of above 100 Hz band signal can be significantly different for orientations and stimulus conditions simultaneously. In area V1, this difference was observed in amplitude correlation between 12 and 30 Hz and more than 70 Hz subbands.

CONCLUSIONS

In conclusion, PAC and AAC can be proper features in orientation perception detection. Our results suggest that in both areas, the significant role of high-band and low-band oscillations of LFPs discloses the reliability of these bands and generally LFP signals in mice visual perception.

EEG Signals Classification Related to Visual Objects Using Long Short-Term Memory Network and Nonlinear Interval Type-2 Fuzzy Regression

By gaining insights into how brain activity is encoded and decoded, we enhance our understanding of brain function. This study introduces a method for classifying EEG signals related to visual objects, employing a combination of an LSTM network and nonlinear interval type-2 fuzzy regression (NIT2FR). Here, ResNet is utilized for feature extraction from images, the LSTM network for feature extraction from EEG signals, and NIT2FR for mapping image features to EEG signal features. The application of type-2 fuzzy logic addresses uncertainties arising from EEG signal nonlinearity, noise, limited data sample size, and diverse mental states among participants. The Stanford database was used for implementation, evaluating effectiveness through metrics like classification accuracy, precision, recall, and F1 score. According to the findings, the LSTM network achieved an accuracy of 55.83% in categorizing images using raw EEG data. When compared to other methods like linear type-2, linear/nonlinear type-1 fuzzy, neural network, and polynomial regression, NIT2FR coupled with an SVM classifier outperformed with a 68.05% accuracy. Thus, NIT2FR demonstrates superiority in handling high uncertainty environments. Moreover, the 6.03% improvement in accuracy over the best previous study using the same dataset underscores its effectiveness. Precision, recall, and F1 score results for NIT2FR were 68.93%, 68.08%, and 68.49% respectively, surpassing outcomes from linear type-2, linear/nonlinear type-1 fuzzy regression methods.

NeuroDM: Decoding and visualizing human brain activity with EEG-guided diffusion model

BACKGROUND AND OBJECTIVE

Brain-Computer Interface (BCI) technology has recently been advancing rapidly, bringing significant hope for improving human health and quality of life. Decoding and visualizing visually evoked electroencephalography (EEG) signals into corresponding images plays a crucial role in the practical application of BCI technology. The recent emergence of diffusion models provides a good modeling basis for this work. However, the existing diffusion models still have great challenges in generating high-quality images from EEG, due to the low signal-to-noise ratio and strong randomness of EEG signals. The purpose of this study is to address the above-mentioned challenges by proposing a framework named NeuroDM that can decode human brain responses to visual stimuli from EEG-recorded brain activity.

METHODS

In NeuroDM, an EEG-Visual-Transformer (EV-Transformer) is used to extract the visual-related features with high classification accuracy from EEG signals, then an EEG-Guided Diffusion Model (EG-DM) is employed to synthesize high-quality images from the EEG visual-related features.

RESULTS

We conducted experiments on two EEG datasets (one is a forty-class dataset, and the other is a four-class dataset). In the task of EEG decoding, we achieved average accuracies of 99.80% and 92.07% on two datasets, respectively. In the task of EEG visualization, the Inception Score of the images generated by NeuroDM reached 15.04 and 8.67, respectively. All the above results outperform existing methods.

CONCLUSIONS

The experimental results on two EEG datasets demonstrate the effectiveness of the NeuroDM framework, achieving state-of-the-art performance in terms of classification accuracy and image quality. Furthermore, our NeuroDM exhibits strong generalization capabilities and the ability to generate diverse images.

Re-emergent Tremor in Parkinson's Disease: Evidence of Pathologic β and Prokinetic γ Activity

BACKGROUND

Re-emergent tremor is characterized as a continuation of resting tremor and is often highly therapy refractory. This study examines variations in brain activity and oscillatory responses between resting and re-emergent tremors in Parkinson's disease.

METHODS

Forty patients with Parkinson's disease (25 males, mean age, 66.78 ± 5.03 years) and 40 age- and sex-matched healthy controls were included in the study. Electroencephalogram and electromyography signals were simultaneously recorded during resting and re-emergent tremors in levodopa on and off states for patients and mimicked by healthy controls. Brain activity was localized using the beamforming technique, and information flow between sources was estimated using effective connectivity. Cross-frequency coupling was used to assess neuronal oscillations between tremor frequency and canonical frequency oscillations.

RESULTS

During levodopa on, differences in brain activity were observed in the premotor cortex and cerebellum in both the patient and control groups. However, Parkinson's disease patients also exhibited additional activity in the primary sensorimotor cortex. On withdrawal of levodopa, different source patterns were observed in the supplementary motor area and basal ganglia area. Additionally, levodopa was found to suppress the strength of connectivity (P < 0.001) between the identified sources and influence the tremor frequency-related coupling, leading to a decrease in β (P < 0.001) and an increase in γ frequency coupling (P < 0.001).

CONCLUSIONS

Distinct variations in cortical-subcortical brain activity are evident in tremor phenotypes. The primary sensorimotor cortex plays a crucial role in the generation of re-emergent tremor. Moreover, oscillatory neuronal responses in pathological β and prokinetic γ activity are specific to tremor phenotypes. © 2024 The Authors. Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society.

Beyond the brain-computer interface: Decoding brain activity as a tool to understand neuronal mechanisms subtending cognition and behavior

One of the major challenges in system neurosciences consists in developing techniques for estimating the cognitive information content in brain activity. This has an enormous potential in different domains spanning from clinical applications, cognitive enhancement to a better understanding of the neural bases of cognition. In this context, the inclusion of machine learning techniques to decode different aspects of human cognition and behavior and its use to develop brain-computer interfaces for applications in neuroprosthetics has supported a genuine revolution in the field. However, while these approaches have been shown quite successful for the study of the motor and sensory functions, success is still far from being reached when it comes to covert cognitive functions such as attention, motivation and decision making. While improvement in this field of BCIs is growing fast, a new research focus has emerged from the development of strategies for decoding neural activity. In this review, we aim at exploring how the advanced in decoding of brain activity is becoming a major neuroscience tool moving forward our understanding of brain functions, providing a robust theoretical framework to test predictions on the relationship between brain activity and cognition and behavior.

Measuring Phase-Amplitude Coupling between Neural Oscillations of Different Frequencies via the Wasserstein Distance

BACKGROUND

Phase-amplitude coupling (PAC) is a key neuronal mechanism. Here, a novel method for quantifying PAC via the Wasserstein distance is presented.

NEW METHOD

The Wasserstein distance is an optimization algorithm for minimizing transportation cost and distance. For the first time, the author has applied this distance function to quantify PAC and named the Wasserstein Modulation Index (wMI). As the wMI accommodates the product of the amplitude value in each phase position and the coupling phase position, it allows for extraction of more detailed PAC features from the data.

RESULTS

The validity of the wMI calculations was examined using various simulation data, including sinusoidal and non-sinusoidal waves and empirical data sets. The current findings showed that the wMI is a more robust and stable index for quantifying PAC under various measuring conditions. Specifically, it can better reflect the timing of coupling and distinguish the shape of the coupling distribution than other measurements, both of which are the most significant parameters related to the functionality of PAC. Furthermore, the wMI is also suitable for many applications, such as more data-driven approaches and direct comparisons.

COMPARISON WITH EXISTING METHOD(S)

Compared with Euler-based PAC methods and the MI, the wMI is not easily affected by the non-sinusoidal nature of neural oscillation and the short data length and enables better reflection of the natures of PAC, such as the timing of coupling and the amplitude distribution in the phase plane, than the MI.

CONCLUSION

The wMI is expected to extract more detailed PAC characteristics, which could considerably contribute to the neuroscience field.

A dual-channel language decoding from brain activity with progressive transfer training

When we view a scene, the visual cortex extracts and processes visual information in the scene through various kinds of neural activities. Previous studies have decoded the neural activity into single/multiple semantic category tags which can caption the scene to some extent. However, these tags are isolated words with no grammatical structure, insufficiently conveying what the scene contains. It is well‐known that textual language (sentences/phrases) is superior to single word in disclosing the meaning of images as well as reflecting people's real understanding of the images. Here, based on artificial intelligence technologies, we attempted to build a dual‐channel language decoding model (DC‐LDM) to decode the neural activities evoked by images into language (phrases or short sentences). The DC‐LDM consisted of five modules, namely, Image‐Extractor, Image‐Encoder, Nerve‐Extractor, Nerve‐Encoder, and Language‐Decoder. In addition, we employed a strategy of progressive transfer to train the DC‐LDM for improving the performance of language decoding. The results showed that the texts decoded by DC‐LDM could describe natural image stimuli accurately and vividly. We adopted six indexes to quantitatively evaluate the difference between the decoded texts and the annotated texts of corresponding visual images, and found that Word2vec‐Cosine similarity (WCS) was the best indicator to reflect the similarity between the decoded and the annotated texts. In addition, among different visual cortices, we found that the text decoded by the higher visual cortex was more consistent with the description of the natural image than the lower one. Our decoding model may provide enlightenment in language‐based brain‐computer interface explorations.

Measuring Phase-Amplitude Coupling Based on the Jensen-Shannon Divergence and Correlation Matrix

Phase-amplitude coupling (PAC) measures the relationship between the phase of low-frequency oscillations (LFO) and the amplitude of high-frequency oscillations (HFO). It plays an important functional role in neural information processing and cognition. Thus, we propose a novel method based on the Jensen-Shannon (JS) divergence and correlation matrix. The method takes the amplitude distributions of the HFO located in the corresponding phase bins of the LFO as multichannel inputs to construct a correlation matrix, where the elements are calculated based on the JS divergence between pairs of amplitude distributions. Then, the omega complexity extracted from the correlation matrix is used to estimate the PAC strength. The simulation results demonstrate that the proposed method can effectively reflect the PAC strength and slightly vary with the data length. Moreover, it outperforms five frequently used algorithms in the performance against additive white Gaussian noise and spike noise and the ability of detecting PAC in wide frequency ranges. To validate our proposed method with real data, it was applied to analyze the local field potential recorded from the dorsomedial striatum in a male Sprague-Dawley rat. It can replicate previous results obtained with other PAC metrics. In conclusion, these results suggest that our proposed method is a powerful tool for measuring the PAC between neural oscillations.

Long short-term memory-based neural decoding of object categories evoked by natural images

Visual perceptual decoding is one of the important and challenging topics in cognitive neuroscience. Building a mapping model between visual response signals and visual contents is the key point of decoding. Most previous studies used peak response signals to decode object categories. However, brain activities measured by functional magnetic resonance imaging are a dynamic process with time dependence, so peak signals cannot fully represent the whole process, which may affect the performance of decoding. Here, we propose a decoding model based on long short-term memory (LSTM) network to decode five object categories from multitime response signals evoked by natural images. Experimental results show that the average decoding accuracy using the multitime (2-6 s) response signals is 0.540 from the five subjects, which is significantly higher than that using the peak ones (6 s; accuracy: 0.492; p < .05). In addition, from the perspective of different durations, methods and visual areas, the decoding performances of the five object categories are deeply and comprehensively explored. The analysis of different durations and decoding methods reveals that the LSTM-based decoding model with sequence simulation ability can fit the time dependence of the multitime visual response signals to achieve higher decoding performance. The comparative analysis of different visual areas demonstrates that the higher visual cortex (VC) contains more semantic category information needed for visual perceptual decoding than lower VC.

The dynamic properties of a brain network during working memory based on the algorithm of cross-frequency coupling

Working memory (WM) refers to a memory system with limited energy for short-term maintenance and plays an important role in cognitive functions. At present, research regarding WM mostly focuses on the coordination between neural signals in the signal microelectrode channel. However, how neural signals coordinate the coding of WM at the network level is rarely studied. Cross-frequency coupling (CFC) reflects the coordinated effect between different frequency components (e.g., theta and gamma) of local field potentials (LFPs) during WM. In this study, we try to map the changes that occur in the brain networks during WM at the level of CFC between theta-gamma of LFPs. First, a 16-channel brain network by using the CFC between theta-gamma of LFPs during WM was constructed. Then, the dynamic properties of the brain network during WM were analyzed based on graph theory. Experimental results show that the LFPs power increased at the WM state than at resting stat, but decreased across learning; the CFC between theta-gamma increased with learning days and phase-amplitude coupling (PAC) in the WM state was higher than that in free choice state and rest state; the changes of average degree, average shortest path length and global efficiency had significant difference on learning days. We can indicate that the CFC between theta-gamma in the network plays an important role in the WM formation. Furthermore, correct storage of WM information will not change local information transmission and the small-world attribute, while, it can increase the network connection and efficiency of information transmission.

Computer Aided Diagnosis System for multiple sclerosis disease based on phase to amplitude coupling in covert visual attention

BACKGROUND AND OBJECTIVE

Computer Aided Diagnosis (CAD) techniques have widely been used in research to detect the neurological abnormalities and improve the consistency of diagnosis and treatment in medicine. In this study, a new CAD system based on EEG signals was developed. The motivation for the development of the CAD system was to diagnose multiple sclerosis (MS) disease during covert visual attention tasks. It is worth noting that research of this kind on the efficacy of attention tasks is limited in scope for MS patients; therefore, it is vital to develop a feature of EEG to characterize the patient's state with high sensitivity and specificity.

METHODS

We evaluated the use of phase-amplitude coupling (PAC) of EEG signals to diagnose MS. It is assumed that the role of PAC for information encoding during visual attention in MS is greatly unknown; therefore, we made an attempt to investigate it via CAD systems. The EEG signals were recorded from healthy and MS patients while performing new visual attention tasks. Machine learning algorithms were also used to identify the EEG signals as to whether the disease existed or not. The challenge regarding the dimensionality of the extracted features was addressed through selecting the relevant and efficient features using T-test and Bhattacharyya distance criteria, and the validity of the system was assessed through leave-one-subject-out cross-validation method.

RESULTS

Our findings indicated that online sequential extreme learning machine (OS-ELM) classifier with T-test feature selection method yielded peak accuracy, sensitivity and specificity in both color and direction tasks. These values were 91%, 83% and 96% for color task, and 90%, 82% and 96% for the direction task.

CONCLUSIONS

Based on the results, it can be concluded that this procedure can be used for the automatic diagnosis of early MS, and can also facilitate the treatment assessment in patients.

Decoding English Alphabet Letters Using EEG Phase Information

Increasing evidence indicates that the phase pattern and power of the low frequency oscillations of brain electroencephalograms (EEG) contain significant information during the human cognition of sensory signals such as auditory and visual stimuli. Here, we investigate whether and how the letters of the alphabet can be directly decoded from EEG phase and power data. In addition, we investigate how different band oscillations contribute to the classification and determine the critical time periods. An English letter recognition task was assigned, and statistical analyses were conducted to decode the EEG signal corresponding to each letter visualized on a computer screen. We applied support vector machine (SVM) with gradient descent method to learn the potential features for classification. It was observed that the EEG phase signals have a higher decoding accuracy than the oscillation power information. Low-frequency theta and alpha oscillations have phase information with higher accuracy than do other bands. The decoding performance was best when the analysis period began from 180 to 380 ms after stimulus presentation, especially in the lateral occipital and posterior temporal scalp regions (PO7 and PO8). These results may provide a new approach for brain-computer interface techniques (BCI) and may deepen our understanding of EEG oscillations in cognition.