MEG Sensor Selection for Neural Speech Decoding

Facilitating cognitive neuroscience research with 80-sensor optically pumped magnetometer magnetoencephalography (OPM-MEG)

Recent advancements in optically pumped magnetometer magnetoencephalography (OPM-MEG) make it a promising alternative to conventional SQUID-MEG systems. Nonetheless, as reported in the literature, current OPM-MEG systems are often constrained by a limited number of sampling points, which restricts their capability to match the full-head coverage offered by SQUID-MEG systems. Additionally, whether OPM-MEG can deliver results comparable to SQUID-MEG in practical cognitive neuroscience applications remains largely unexplored. In this study, we introduce a high-density, full-head coverage OPM-MEG system with 80 sensors and systematically compare the performance of OPM-MEG and SQUID-MEG, from sensor- to source-level analysis, across various classic cognitive tasks. Our results demonstrate that visual and auditory evoked fields captured using OPM-MEG align closely with those obtained from SQUID-MEG. Furthermore, steady-state visual evoked field and finger-tapping-induced beta power change recorded with OPM-MEG are accurately localized to corresponding brain regions, with activation centers highly congruent to those observed with SQUID-MEG. For resting-state recordings, the two modalities exhibit similar power distributions, functional connectomes, and microstate clusters. These findings indicate that the 80-sensor OPM-MEG system provides spatial and temporal characteristics comparable to those of traditional SQUID-MEG. Thus, our study offers empirical evidence supporting the efficacy of high-density OPM-MEG and suggests that OPM-MEG, with dense sampling capability, represents a compelling alternative to conventional SQUID-MEG, facilitating further exploration of human cognition.

Signal acquisition of brain-computer interfaces: A medical-engineering crossover perspective review

Brain-computer interface (BCI) technology represents a burgeoning interdisciplinary domain that facilitates direct communication between individuals and external devices. The efficacy of BCI systems is largely contingent upon the progress in signal acquisition methodologies. This paper endeavors to provide an exhaustive synopsis of signal acquisition technologies within the realm of BCI by scrutinizing research publications from the last ten years. Our review synthesizes insights from both clinical and engineering viewpoints, delineating a comprehensive two-dimensional framework for understanding signal acquisition in BCIs. We delineate nine discrete categories of technologies, furnishing exemplars for each and delineating the salient challenges pertinent to these modalities. This review furnishes researchers and practitioners with a broad-spectrum comprehension of the signal acquisition landscape in BCI, and deliberates on the paramount issues presently confronting the field. Prospective enhancements in BCI signal acquisition should focus on harmonizing a multitude of disciplinary perspectives. Achieving equilibrium between signal fidelity, invasiveness, biocompatibility, and other pivotal considerations is imperative. By doing so, we can propel BCI technology forward, bolstering its effectiveness, safety, and dependability, thereby contributing to an auspicious future for human-technology integration.

Coherence-based channel selection and Riemannian geometry features for magnetoencephalography decoding

Magnetoencephalography (MEG) records the extremely weak magnetic fields on the surface of the scalp through highly sensitive sensors. Multi-channel MEG data provide higher spatial and temporal resolution when measuring brain activities, and can be applied for brain-computer interfaces as well. However, a large number of channels leads to high computational complexity and can potentially impact decoding accuracy. To improve the accuracy of MEG decoding, this paper proposes a new coherence-based channel selection method that effectively identifies task-relevant channels, reducing the presence of noisy and redundant information. Riemannian geometry is then used to extract effective features from selected channels of MEG data. Finally, MEG decoding is achieved by training a support vector machine classifier with the Radial Basis Function kernel. Experiments were conducted on two public MEG datasets to validate the effectiveness of the proposed method. The results from Dataset 1 show that Riemannian geometry achieves higher classification accuracy (compared to common spatial patterns and power spectral density) in the single-subject visual decoding task. Moreover, coherence-based channel selection significantly (P = 0.0002) outperforms the use of all channels. Moving on to Dataset 2, the results reveal that coherence-based channel selection is also significantly (P <0.0001) superior to all channels and channels around C3 and C4 in cross-session mental imagery decoding tasks. Additionally, the proposed method outperforms state-of-the-art methods in motor imagery tasks.

Artificial Intelligence and the Future of Communication Sciences and Disorders: A Bibliometric and Visualization Analysis

PURPOSE

As artificial intelligence (AI) takes an increasingly prominent role in health care, a growing body of research is being dedicated to its application in the investigation of communication sciences and disorders (CSD). This study aims to provide a comprehensive overview, serving as a valuable resource for researchers, developers, and professionals seeking to comprehend the evolving landscape of AI in CSD research.

METHOD

We conducted a bibliometric analysis of AI-based research in the discipline of CSD published up to December 2023. Utilizing the Web of Science and Scopus databases, we identified 15,035 publications, with 4,375 meeting our inclusion criteria. Based on the bibliometric data, we examined publication trends and patterns, characteristics of research activities, and research hotspot tendencies.

RESULTS

From 1985 onwards, there has been a consistent annual increase in publications, averaging 16.51%, notably surging from 2012 to 2023. The primary communication disorders studied include autism, aphasia, dysarthria, Parkinson's disease, and Alzheimer's disease. Noteworthy AI models instantiated in CSD research encompass support vector machine, convolutional neural network, and hidden Markov model, among others.

CONCLUSIONS

Compared to AI applications in other fields, the adoption of AI in CSD has lagged slightly behind. While CSD studies primarily use classical machine learning techniques, there is a growing trend toward the integration of deep learning methods. AI technology offers significant benefits for both research and clinical practice in CSD, but it also presents certain challenges. Moving forward, collaboration among technological, research, and clinical domains is essential to empower researchers and speech-language pathologists to effectively leverage AI technology for the study, diagnosis, assessment, and rehabilitation of CSD.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.27162564.

Decoding Brain Signals from Rapid-Event EEG for Visual Analysis Using Deep Learning

The perception and recognition of objects around us empower environmental interaction. Harnessing the brain's signals to achieve this objective has consistently posed difficulties. Researchers are exploring whether the poor accuracy in this field is a result of the design of the temporal stimulation (block versus rapid event) or the inherent complexity of electroencephalogram (EEG) signals. Decoding perceptive signal responses in subjects has become increasingly complex due to high noise levels and the complex nature of brain activities. EEG signals have high temporal resolution and are non-stationary signals, i.e., their mean and variance vary overtime. This study aims to develop a deep learning model for the decoding of subjects' responses to rapid-event visual stimuli and highlights the major factors that contribute to low accuracy in the EEG visual classification task.The proposed multi-class, multi-channel model integrates feature fusion to handle complex, non-stationary signals. This model is applied to the largest publicly available EEG dataset for visual classification consisting of 40 object classes, with 1000 images in each class. Contemporary state-of-the-art studies in this area investigating a large number of object classes have achieved a maximum accuracy of 17.6%. In contrast, our approach, which integrates Multi-Class, Multi-Channel Feature Fusion (MCCFF), achieves a classification accuracy of 33.17% for 40 classes. These results demonstrate the potential of EEG signals in advancing EEG visual classification and offering potential for future applications in visual machine models.

Wireless optically pumped magnetometer MEG

The current magnetoencephalography (MEG) systems, which rely on cables for control and signal transmission, do not fully realize the potential of wearable optically pumped magnetometers (OPM). This study presents a significant advancement in wireless OPM-MEG by reducing magnetization in the electronics and developing a tailored wireless communication protocol. Our protocol effectively eliminates electromagnetic interference, particularly in the critical frequency bands of MEG signals, and accurately synchronizes the acquisition and stimulation channels with the host computer's clock. We have successfully achieved single-channel wireless OPM-MEG measurement and demonstrated its reliability by replicating three well-established experiments: The alpha rhythm, auditory evoked field, and steady-state visual evoked field in the human brain. Our prototype wireless OPM-MEG system not only streamlines the measurement process but also represents a major step forward in the development of wearable OPM-MEG applications in both neuroscience and clinical research.

Decoding N400m Evoked Component: A Tutorial on Multivariate Pattern Analysis for OP-MEG Data

Multivariate pattern analysis (MVPA) has played an extensive role in interpreting brain activity, which has been applied in studies with modalities such as functional Magnetic Resonance Imaging (fMRI), Magnetoencephalography (MEG) and Electroencephalography (EEG). The advent of wearable MEG systems based on optically pumped magnetometers (OPMs), i.e., OP-MEG, has broadened the application of bio-magnetism in the realm of neuroscience. Nonetheless, it also raises challenges in temporal decoding analysis due to the unique attributes of OP-MEG itself. The efficacy of decoding performance utilizing multimodal fusion, such as MEG-EEG, also remains to be elucidated. In this regard, we investigated the impact of several factors, such as processing methods, models and modalities, on the decoding outcomes of OP-MEG. Our findings indicate that the number of averaged trials, dimensionality reduction (DR) methods, and the number of cross-validation folds significantly affect the decoding performance of OP-MEG data. Additionally, decoding results vary across modalities and fusion strategy. In contrast, decoder type, resampling frequency, and sliding window length exert marginal effects. Furthermore, we introduced mutual information (MI) to investigate how information loss due to OP-MEG data processing affect decoding accuracy. Our study offers insights for linear decoding research using OP-MEG and expand its application in the fields of cognitive neuroscience.

Decoding kinematic information from beta-band motor rhythms of speech motor cortex: a methodological/analytic approach using concurrent speech movement tracking and magnetoencephalography

Introduction

Articulography and functional neuroimaging are two major tools for studying the neurobiology of speech production. Until now, however, it has generally not been feasible to use both in the same experimental setup because of technical incompatibilities between the two methodologies.

Methods

Here we describe results from a novel articulography system dubbed Magneto-articulography for the Assessment of Speech Kinematics (MASK), which is technically compatible with magnetoencephalography (MEG) brain scanning systems. In the present paper we describe our methodological and analytic approach for extracting brain motor activities related to key kinematic and coordination event parameters derived from time-registered MASK tracking measurements. Data were collected from 10 healthy adults with tracking coils on the tongue, lips, and jaw. Analyses targeted the gestural landmarks of reiterated utterances/ipa/ and /api/, produced at normal and faster rates.

Results

The results show that (1) Speech sensorimotor cortex can be reliably located in peri-rolandic regions of the left hemisphere; (2) mu (8-12 Hz) and beta band (13-30 Hz) neuromotor oscillations are present in the speech signals and contain information structures that are independent of those present in higher-frequency bands; and (3) hypotheses concerning the information content of speech motor rhythms can be systematically evaluated with multivariate pattern analytic techniques.

Discussion

These results show that MASK provides the capability, for deriving subject-specific articulatory parameters, based on well-established and robust motor control parameters, in the same experimental setup as the MEG brain recordings and in temporal and spatial co-register with the brain data. The analytic approach described here provides new capabilities for testing hypotheses concerning the types of kinematic information that are encoded and processed within specific components of the speech neuromotor system.

Bimodal electroencephalography-functional magnetic resonance imaging dataset for inner-speech recognition

The recognition of inner speech, which could give a 'voice' to patients that have no ability to speak or move, is a challenge for brain-computer interfaces (BCIs). A shortcoming of the available datasets is that they do not combine modalities to increase the performance of inner speech recognition. Multimodal datasets of brain data enable the fusion of neuroimaging modalities with complimentary properties, such as the high spatial resolution of functional magnetic resonance imaging (fMRI) and the temporal resolution of electroencephalography (EEG), and therefore are promising for decoding inner speech. This paper presents the first publicly available bimodal dataset containing EEG and fMRI data acquired nonsimultaneously during inner-speech production. Data were obtained from four healthy, right-handed participants during an inner-speech task with words in either a social or numerical category. Each of the 8-word stimuli were assessed with 40 trials, resulting in 320 trials in each modality for each participant. The aim of this work is to provide a publicly available bimodal dataset on inner speech, contributing towards speech prostheses.

Slow Firing Single Units Are Essential for Optimal Decoding of Silent Speech

The motivation of someone who is locked-in, that is, paralyzed and mute, is to find relief for their loss of function. The data presented in this report is part of an attempt to restore one of those lost functions, namely, speech. An essential feature of the development of a speech prosthesis is optimal decoding of patterns of recorded neural signals during silent or covert speech, that is, speaking “inside the head” with output that is inaudible due to the paralysis of the articulators. The aim of this paper is to illustrate the importance of both fast and slow single unit firings recorded from an individual with locked-in syndrome and from an intact participant speaking silently. Long duration electrodes were implanted in the motor speech cortex for up to 13 years in the locked-in participant. The data herein provide evidence that slow firing single units are essential for optimal decoding accuracy. Additional evidence indicates that slow firing single units can be conditioned in the locked-in participant 5 years after implantation, further supporting their role in decoding.

Rethinking the Methods and Algorithms for Inner Speech Decoding and Making Them Reproducible

This study focuses on the automatic decoding of inner speech using noninvasive methods, such as Electroencephalography (EEG). While inner speech has been a research topic in philosophy and psychology for half a century, recent attempts have been made to decode nonvoiced spoken words by using various brain-computer interfaces. The main shortcomings of existing work are reproducibility and the availability of data and code. In this work, we investigate various methods (using Convolutional Neural Network (CNN), Gated Recurrent Unit (GRU), Long Short-Term Memory Networks (LSTM)) for the detection task of five vowels and six words on a publicly available EEG dataset. The main contributions of this work are (1) subject dependent vs. subject-independent approaches, (2) the effect of different preprocessing steps (Independent Component Analysis (ICA), down-sampling and filtering), and (3) word classification (where we achieve state-of-the-art performance on a publicly available dataset). Overall we achieve a performance accuracy of 35.20% and 29.21% when classifying five vowels and six words, respectively, in a publicly available dataset, using our tuned iSpeech-CNN architecture. All of our code and processed data are publicly available to ensure reproducibility. As such, this work contributes to a deeper understanding and reproducibility of experiments in the area of inner speech detection.

Imagined speech can be decoded from low- and cross-frequency intracranial EEG features

Reconstructing intended speech from neural activity using brain-computer interfaces holds great promises for people with severe speech production deficits. While decoding overt speech has progressed, decoding imagined speech has met limited success, mainly because the associated neural signals are weak and variable compared to overt speech, hence difficult to decode by learning algorithms. We obtained three electrocorticography datasets from 13 patients, with electrodes implanted for epilepsy evaluation, who performed overt and imagined speech production tasks. Based on recent theories of speech neural processing, we extracted consistent and specific neural features usable for future brain computer interfaces, and assessed their performance to discriminate speech items in articulatory, phonetic, and vocalic representation spaces. While high-frequency activity provided the best signal for overt speech, both low- and higher-frequency power and local cross-frequency contributed to imagined speech decoding, in particular in phonetic and vocalic, i.e. perceptual, spaces. These findings show that low-frequency power and cross-frequency dynamics contain key information for imagined speech decoding.

Magnetometers vs Gradiometers for Neural Speech Decoding

Neural speech decoding aims at providing natural rate communication assistance to patients with locked-in state (e.g. due to amyotrophic lateral sclerosis, ALS) in contrast to the traditional brain-computer interface (BCI) spellers which are slow. Recent studies have shown that Magnetoencephalography (MEG) is a suitable neuroimaging modality to study neural speech decoding considering its excellent temporal resolution that can characterize the fast dynamics of speech. Gradiometers have been the preferred choice for sensor space analysis with MEG, due to their efficacy in noise suppression over magnetometers. However, recent development of optically pumped magnetometers (OPM) based wearable-MEG devices have shown great potential in future BCI applications, yet, no prior study has evaluated the performance of magnetometers in neural speech decoding. In this study, we decoded imagined and spoken speech from the MEG signals of seven healthy participants and compared the performance of magnetometers and gradiometers. Experimental results indicated that magnetometers also have the potential for neural speech decoding, although the performance was significantly lower than that obtained with gradiometers. Further, we implemented a wavelet based denoising strategy that improved the performance of both magnetometers and gradiometers significantly. These findings reconfirm that gradiometers are preferable in MEG based decoding analysis but also provide the possibility towards the use of magnetometers (or OPMs) for the development of the next-generation speech-BCIs.

Decoding Imagined Speech Based on Deep Metric Learning for Intuitive BCI Communication

Imagined speech is a highly promising paradigm due to its intuitive application and multiclass scalability in the field of brain-computer interfaces. However, optimal feature extraction and classifiers have not yet been established. Furthermore, retraining still requires a large number of trials when new classes are added. The aim of this study is (i) to increase the classification performance for imagined speech and (ii) to apply a new class using a pretrained classifier with a small number of trials. We propose a novel framework based on deep metric learning that learns the distance by comparing the similarity between samples. We also applied the instantaneous frequency and spectral entropy used for speech signals to electroencephalography signals during imagined speech. The method was evaluated on two public datasets (6-class Coretto DB and 5-class BCI Competition DB). We achieved a 6-class accuracy of 45.00 ± 3.13% and a 5-class accuracy of 48.10 ± 3.68% using the proposed method, which significantly outperformed state-of-the-art methods. Additionally, we verified that the new class could be detected through incremental learning with a small number of trials. As a result, the average accuracy is 44.50 ± 0.26% for Coretto DB and 47.12 ± 0.27% for BCI Competition DB, which shows similar accuracy to baseline accuracy without incremental learning. Our results have shown that the accuracy can be greatly improved even with a small number of trials by selecting appropriate features from imagined speech. The proposed framework could be directly used to help construct an extensible intuitive communication system based on brain-computer interfaces.

Decoding Covert Speech From EEG-A Comprehensive Review

Over the past decade, many researchers have come up with different implementations of systems for decoding covert or imagined speech from EEG (electroencephalogram). They differ from each other in several aspects, from data acquisition to machine learning algorithms, due to which, a comparison between different implementations is often difficult. This review article puts together all the relevant works published in the last decade on decoding imagined speech from EEG into a single framework. Every important aspect of designing such a system, such as selection of words to be imagined, number of electrodes to be recorded, temporal and spatial filtering, feature extraction and classifier are reviewed. This helps a researcher to compare the relative merits and demerits of the different approaches and choose the one that is most optimal. Speech being the most natural form of communication which human beings acquire even without formal education, imagined speech is an ideal choice of prompt for evoking brain activity patterns for a BCI (brain-computer interface) system, although the research on developing real-time (online) speech imagery based BCI systems is still in its infancy. Covert speech based BCI can help people with disabilities to improve their quality of life. It can also be used for covert communication in environments that do not support vocal communication. This paper also discusses some future directions, which will aid the deployment of speech imagery based BCI for practical applications, rather than only for laboratory experiments.