Brain-Based Binary Communication Using Spatiotemporal Features of fNIRS Responses

Novel Feature Generation for Classification of Motor Activity from Functional Near-Infrared Spectroscopy Signals Using Machine Learning

Recent research in the field of cognitive motor action decoding focuses on data acquired from Functional Near-Infrared Spectroscopy (fNIRS) and its analysis. This research aims to classify two different motor activities, namely, mental drawing (MD) and spatial navigation (SN), using fNIRS data from non-motor baseline data and other motor activities. Accurate activity detection in non-stationary signals like fNIRS is challenging and requires complex feature descriptors. As a novel framework, a new feature generation by fusion of wavelet feature, Hilbert, symlet, and Hjorth parameters is proposed for improving the accuracy of the classification. This new fused feature has statistical descriptor elements, time-localization in the frequency domain, edge feature, texture features, and phase information to detect and locate the activity accurately. Three types of independent component analysis, including FastICA, Picard, and Infomax were implemented for preprocessing which removes noises and motion artifacts. Two independent binary classifiers are designed to handle the complexity of classification in which one is responsible for mental drawing (MD) detection and the other one is spatial navigation (SN). Four different types of algorithms including nearest neighbors (KNN), Linear Discriminant Analysis (LDA), light gradient-boosting machine (LGBM), and Extreme Gradient Boosting (XGBOOST) were implemented. It has been identified that the LGBM classifier gives high accuracies-98% for mental drawing and 97% for spatial navigation. Comparison with existing research proves that the proposed method gives the highest classification accuracies. Statistical validation of the proposed new feature generation by the Kruskal-Wallis H-test and Mann-Whitney U non-parametric test proves the reliability of the proposed mechanism.

Boosting brain-computer interfaces with functional electrical stimulation: potential applications in people with locked-in syndrome

Individuals with a locked-in state live with severe whole-body paralysis that limits their ability to communicate with family and loved ones. Recent advances in brain-computer interface (BCI) technology have presented a potential alternative for these people to communicate by detecting neural activity associated with attempted hand or speech movements and translating the decoded intended movements to a control signal for a computer. A technique that could potentially enrich the communication capacity of BCIs is functional electrical stimulation (FES) of paralyzed limbs and face to restore body and facial movements of paralyzed individuals, allowing to add body language and facial expression to communication BCI utterances. Here, we review the current state of the art of existing BCI and FES work in people with paralysis of body and face and propose that a combined BCI-FES approach, which has already proved successful in several applications in stroke and spinal cord injury, can provide a novel promising mode of communication for locked-in individuals.

It takes two (seconds): decreasing encoding time for two-choice functional near-infrared spectroscopy brain-computer interface communication

Significance

Brain-computer interfaces (BCIs) can provide severely motor-impaired patients with a motor-independent communication channel. Functional near-infrared spectroscopy (fNIRS) constitutes a promising BCI-input modality given its high mobility, safety, user comfort, cost-efficiency, and relatively low motion sensitivity.

Aim

The present study aimed at developing an efficient and convenient two-choice fNIRS communication BCI by implementing a relatively short encoding time (2 s), considerably increasing communication speed, and decreasing the cognitive load of BCI users.

Approach

To encode binary answers to 10 biographical questions, 10 healthy adults repeatedly performed a combined motor-speech imagery task within 2 different time windows guided by auditory instructions. Each answer-encoding run consisted of 10 trials. Answers were decoded during the ongoing experiment from the time course of the individually identified most-informative fNIRS channel-by-chromophore combination.

Results

The answers of participants were decoded online with an accuracy of 85.8% (run-based group mean). Post-hoc analysis yielded an average single-trial accuracy of 68.1%. Analysis of the effect of number of trial repetitions showed that the best information-transfer rate could be obtained by combining four encoding trials.

Conclusions

The study demonstrates that an encoding time as short as 2 s can enable immediate, efficient, and convenient fNIRS-BCI communication.

Talking with hands and feet: Selective somatosensory attention and fMRI enable robust and convenient brain-based communication

In brain-based communication, voluntarily modulated brain signals (instead of motor output) are utilized to interact with the outside world. The possibility to circumvent the motor system constitutes an important alternative option for severely paralyzed. Most communication brain-computer interface (BCI) paradigms require intact visual capabilities and impose a high cognitive load, but for some patients, these requirements are not given. In these situations, a better-suited, less cognitively demanding information-encoding approach may exploit auditorily-cued selective somatosensory attention to vibrotactile stimulation. Here, we propose, validate and optimize a novel communication-BCI paradigm using differential fMRI activation patterns evoked by selective somatosensory attention to tactile stimulation of the right hand or left foot. Using cytoarchitectonic probability maps and multi-voxel pattern analysis (MVPA), we show that the locus of selective somatosensory attention can be decoded from fMRI-signal patterns in (especially primary) somatosensory cortex with high accuracy and reliability, with the highest classification accuracy (85.93%) achieved when using Brodmann area 2 (SI-BA2) at a probability level of 0.2. Based on this outcome, we developed and validated a novel somatosensory attention-based yes/no communication procedure and demonstrated its high effectiveness even when using only a limited amount of (MVPA) training data. For the BCI user, the paradigm is straightforward, eye-independent, and requires only limited cognitive functioning. In addition, it is BCI-operator friendly given its objective and expertise-independent procedure. For these reasons, our novel communication paradigm has high potential for clinical applications.

Positive Prospective Mental Imagery Characteristics in Young Adults and Their Associations with Depressive Symptoms

Background

Positive prospective mental imagery plays an important role in mental well-being, and depressive symptoms have been associated with difficulties in generating positive prospective mental images (PPMIs). We used a mobile app to gather PPMIs generated by young adults during the COVID-19 pandemic and analyzed content, characteristics, and associations with depressive symptoms.

Methods

This is a secondary analysis of a randomized controlled trial with 95 healthy young adults allocated into two groups (intervention and control). Participants used the mobile app decreasing mental health symptoms for seven consecutive days. Fifty participants in the intervention group reported PPMIs at least three times per day using a mobile app inducing PPMI generation. We categorized entries into themes and applied moderation models to investigate associations between PPMI characteristics and depressive symptoms.

Results

We distinguished 25 PPMI themes. The most frequent were related to consuming food and drinks, watching TV/streaming platforms, and doing sports. Vividness and ease of generation of PPMIs, but not their anticipation, pleasure intensity or number of engagements with the app were associated with fewer depressive symptoms.

Conclusions

We identified PPMI themes in young adults and found significant negative associations between depressive symptoms and vividness and generation ease of PPMIs. These results may inform prevention and intervention science, including the design of personalized interventions. We discuss implications for future studies and treatment development for individuals experiencing diminished PPMI.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10608-023-10378-5.

Interdisciplinary views of fNIRS: Current advancements, equity challenges, and an agenda for future needs of a diverse fNIRS research community

Functional Near-Infrared Spectroscopy (fNIRS) is an innovative and promising neuroimaging modality for studying brain activity in real-world environments. While fNIRS has seen rapid advancements in hardware, software, and research applications since its emergence nearly 30 years ago, limitations still exist regarding all three areas, where existing practices contribute to greater bias within the neuroscience research community. We spotlight fNIRS through the lens of different end-application users, including the unique perspective of a fNIRS manufacturer, and report the challenges of using this technology across several research disciplines and populations. Through the review of different research domains where fNIRS is utilized, we identify and address the presence of bias, specifically due to the restraints of current fNIRS technology, limited diversity among sample populations, and the societal prejudice that infiltrates today's research. Finally, we provide resources for minimizing bias in neuroscience research and an application agenda for the future use of fNIRS that is equitable, diverse, and inclusive.

See, Hear, or Feel - to Speak: A Versatile Multiple-Choice Functional Near-Infrared Spectroscopy-Brain-Computer Interface Feasible With Visual, Auditory, or Tactile Instructions

Severely motor-disabled patients, such as those suffering from the so-called "locked-in" syndrome, cannot communicate naturally. They may benefit from brain-computer interfaces (BCIs) exploiting brain signals for communication and therewith circumventing the muscular system. One BCI technique that has gained attention recently is functional near-infrared spectroscopy (fNIRS). Typically, fNIRS-based BCIs allow for brain-based communication via voluntarily modulation of brain activity through mental task performance guided by visual or auditory instructions. While the development of fNIRS-BCIs has made great progress, the reliability of fNIRS-BCIs across time and environments has rarely been assessed. In the present fNIRS-BCI study, we tested six healthy participants across three consecutive days using a straightforward four-choice fNIRS-BCI communication paradigm that allows answer encoding based on instructions using various sensory modalities. To encode an answer, participants performed a motor imagery task (mental drawing) in one out of four time periods. Answer encoding was guided by either the visual, auditory, or tactile sensory modality. Two participants were tested outside the laboratory in a cafeteria. Answers were decoded from the time course of the most-informative fNIRS channel-by-chromophore combination. Across the three testing days, we obtained mean single- and multi-trial (joint analysis of four consecutive trials) accuracies of 62.5 and 85.19%, respectively. Obtained multi-trial accuracies were 86.11% for visual, 80.56% for auditory, and 88.89% for tactile sensory encoding. The two participants that used the fNIRS-BCI in a cafeteria obtained the best single- (72.22 and 77.78%) and multi-trial accuracies (100 and 94.44%). Communication was reliable over the three recording sessions with multi-trial accuracies of 86.11% on day 1, 86.11% on day 2, and 83.33% on day 3. To gauge the trade-off between number of optodes and decoding accuracy, averaging across two and three promising fNIRS channels was compared to the one-channel approach. Multi-trial accuracy increased from 85.19% (one-channel approach) to 91.67% (two-/three-channel approach). In sum, the presented fNIRS-BCI yielded robust decoding results using three alternative sensory encoding modalities. Further, fNIRS-BCI communication was stable over the course of three consecutive days, even in a natural (social) environment. Therewith, the developed fNIRS-BCI demonstrated high flexibility, reliability and robustness, crucial requirements for future clinical applicability.

The mapping of cortical activation by near-infrared spectroscopy might be a biomarker related to the severity of fibromyalgia symptoms

The delta value of oxyhemoglobin (Δ-HbO) determined by functional near-infrared spectroscopy at prefrontal cortex (PFC) and motor cortex (MC) based on primary (25 °C) and secondary (5 °C) thermal stimuli presented a larger peak latency at left MC in fibromyalgia than in controls. The difference between HbO concentration 15 s after the thermal stimuli ending and HbO concentration before the thermal stimuli onset (Δ-HbO*) at left PFC increased 47.82% in fibromyalgia and 76.66% in controls. This value had satisfactory discriminatory properties to differentiate cortical activation in fibromyalgia versus controls. A receiver operator characteristics (ROC) analysis showed the Δ-HbO* cutoffs of − 0.175 at left PFC and − 0.205 at right PFC offer sensitivity and specificity of at least 80% in screening fibromyalgia from controls. In fibromyalgia, a ROC analysis showed that these cutoffs could discriminate those with higher disability due to pain and more severe central sensitization symptoms (CSS). The ROC with the best discriminatory profile was the CSS score with the Δ-HbO* at left PFC (area under the curve = 0.82, 95% confidence interval = 0.61–100). These results indicate that cortical activation based on Δ-HbO* at left PFC might be a sensitive marker to identify fibromyalgia subjects with more severe clinical symptoms.

The Potential Role of fNIRS in Evaluating Levels of Consciousness

Over the last few decades, neuroimaging techniques have transformed our understanding of the brain and the effect of neurological conditions on brain function. More recently, light-based modalities such as functional near-infrared spectroscopy have gained popularity as tools to study brain function at the bedside. A recent application is to assess residual awareness in patients with disorders of consciousness, as some patients retain awareness albeit lacking all behavioural response to commands. Functional near-infrared spectroscopy can play a vital role in identifying these patients by assessing command-driven brain activity. The goal of this review is to summarise the studies reported on this topic, to discuss the technical and ethical challenges of working with patients with disorders of consciousness, and to outline promising future directions in this field.

Guiding functional near-infrared spectroscopy optode-layout design using individual (f)MRI data: effects on signal strength

Significance: Designing optode layouts is an essential step for functional near-infrared spectroscopy (fNIRS) experiments as the quality of the measured signal and the sensitivity to cortical regions-of-interest depend on how optodes are arranged on the scalp. This becomes particularly relevant for fNIRS-based brain-computer interfaces (BCIs), where developing robust systems with few optodes is crucial for clinical applications. Aim: Available resources often dictate the approach researchers use for optode-layout design. We investigated whether guiding optode layout design using different amounts of subject-specific magnetic resonance imaging (MRI) data affects the fNIRS signal quality and sensitivity to brain activation when healthy participants perform mental-imagery tasks typically used in fNIRS-BCI experiments. Approach: We compared four approaches that incrementally incorporated subject-specific MRI information while participants performed mental-calculation, mental-rotation, and inner-speech tasks. The literature-based approach (LIT) used a literature review to guide the optode layout design. The probabilistic approach (PROB) employed individual anatomical data and probabilistic maps of functional MRI (fMRI)-activation from an independent dataset. The individual fMRI (iFMRI) approach used individual anatomical and fMRI data, and the fourth approach used individual anatomical, functional, and vascular information of the same subject (fVASC). Results: The four approaches resulted in different optode layouts and the more informed approaches outperformed the minimally informed approach (LIT) in terms of signal quality and sensitivity. Further, PROB, iFMRI, and fVASC approaches resulted in a similar outcome. Conclusions: We conclude that additional individual MRI data lead to a better outcome, but that not all the modalities tested here are required to achieve a robust setup. Finally, we give preliminary advice to efficiently using resources for developing robust optode layouts for BCI and neurofeedback applications.