Decoding task engagement from distributed network electrophysiology in humans

Real-time estimation of EEG-based engagement in different tasks

Objective.Recent trends in brain-computer interface (BCI) research concern the passive monitoring of brain activity, which aim to monitor a wide variety of cognitive states. Engagement is such a cognitive state, which is of interest in contexts such as learning, entertainment or rehabilitation. This study proposes a novel approach for real-time estimation of engagement during different tasks using electroencephalography (EEG).Approach.Twenty-three healthy subjects participated in the BCI experiment. A modified version of the d2 test was used to elicit engagement. Within-subject classification models which discriminate between engaging and resting states were trained based on EEG recorded during a d2 test based paradigm. The EEG was recorded using eight electrodes and the classification model was based on filter-bank common spatial patterns and a linear discriminant analysis. The classification models were evaluated in cross-task applications, namely when playing Tetris at different speeds (i.e. slow, medium, fast) and when watching two videos (i.e. advertisement and landscape video). Additionally, subjects' perceived engagement was quantified using a questionnaire.Main results.The models achieved a classification accuracy of 90% on average when tested on an independent d2 test paradigm recording. Subjects' perceived and estimated engagement were found to be greater during the advertisement compared to the landscape video (p= 0.025 andp<0.001, respectively); greater during medium and fast compared to slow Tetris speed (p<0.001, respectively); not different between medium and fast Tetris speeds. Additionally, a common linear relationship was observed for perceived and estimated engagement (rrm= 0.44,p<0.001). Finally, theta and alpha band powers were investigated, which respectively increased and decreased during more engaging states.Significance.This study proposes a task-specific EEG engagement estimation model with cross-task capabilities, offering a framework for real-world applications.

Closing the loop in psychiatric deep brain stimulation: physiology, psychometrics, and plasticity

Deep brain stimulation (DBS) is an invasive approach to precise modulation of psychiatrically relevant circuits. Although it has impressive results in open-label psychiatric trials, DBS has also struggled to scale to and pass through multi-center randomized trials. This contrasts with Parkinson disease, where DBS is an established therapy treating thousands of patients annually. The core difference between these clinical applications is the difficulty of proving target engagement, and of leveraging the wide range of possible settings (parameters) that can be programmed in a given patient's DBS. In Parkinson's, patients' symptoms change rapidly and visibly when the stimulator is tuned to the correct parameters. In psychiatry, those same changes take days to weeks, limiting a clinician's ability to explore parameter space and identify patient-specific optimal settings. I review new approaches to psychiatric target engagement, with an emphasis on major depressive disorder (MDD). Specifically, I argue that better engagement may come by focusing on the root causes of psychiatric illness: dysfunction in specific, measurable cognitive functions and in the connectivity and synchrony of distributed brain circuits. I overview recent progress in both those domains, and how it may relate to other technologies discussed in companion articles in this issue.

Modular pipeline for reconstruction and localization of implanted intracranial ECoG and sEEG electrodes

Implantation of electrodes in the brain has been used as a clinical tool for decades to stimulate and record brain activity. As this method increasingly becomes the standard of care for several disorders and diseases, there is a growing need to quickly and accurately localize the electrodes once they are placed within the brain. We share here a protocol pipeline for localizing electrodes implanted in the brain, which we have applied to more than 260 patients, that is accessible to multiple skill levels and modular in execution. This pipeline uses multiple software packages to prioritize flexibility by permitting multiple different parallel outputs while minimizing the number of steps for each output. These outputs include co-registered imaging, electrode coordinates, 2D and 3D visualizations of the implants, automatic surface and volumetric localizations of the brain regions per electrode, and anonymization and data sharing tools. We demonstrate here some of the pipeline's visualizations and automatic localization algorithms which we have applied to determine appropriate stimulation targets, to conduct seizure dynamics analysis, and to localize neural activity from cognitive tasks in previous studies. Further, the output facilitates the extraction of information such as the probability of grey matter intersection or the nearest anatomic structure per electrode contact across all data sets that go through the pipeline. We expect that this pipeline will be a useful framework for researchers and clinicians alike to localize implanted electrodes in the human brain.

Closed-loop enhancement and neural decoding of cognitive control in humans

Deficits in cognitive control-that is, in the ability to withhold a default pre-potent response in favour of a more adaptive choice-are common in depression, anxiety, addiction and other mental disorders. Here we report proof-of-concept evidence that, in participants undergoing intracranial epilepsy monitoring, closed-loop direct stimulation of the internal capsule or striatum, especially the dorsal sites, enhances the participants' cognitive control during a conflict task. We also show that closed-loop stimulation upon the detection of lapses in cognitive control produced larger behavioural changes than open-loop stimulation, and that task performance for single trials can be directly decoded from the activity of a small number of electrodes via neural features that are compatible with existing closed-loop brain implants. Closed-loop enhancement of cognitive control might remediate underlying cognitive deficits and aid the treatment of severe mental disorders.

Causally mapping human threat extinction relevant circuits with depolarizing brain stimulation methods

Laboratory threat extinction paradigms and exposure-based therapy both involve repeated, safe confrontation with stimuli previously experienced as threatening. This fundamental procedural overlap supports laboratory threat extinction as a compelling analogue of exposure-based therapy. Threat extinction impairments have been detected in clinical anxiety and may contribute to exposure-based therapy non-response and relapse. However, efforts to improve exposure outcomes using techniques that boost extinction - primarily rodent extinction - have largely failed to date, potentially due to fundamental differences between rodent and human neurobiology. In this review, we articulate a comprehensive pre-clinical human research agenda designed to overcome these failures. We describe how connectivity guided depolarizing brain stimulation methods (i.e., TMS and DBS) can be applied concurrently with threat extinction and dual threat reconsolidation-extinction paradigms to causally map human extinction relevant circuits and inform the optimal integration of these methods with exposure-based therapy. We highlight candidate targets including the amygdala, hippocampus, ventromedial prefrontal cortex, dorsal anterior cingulate cortex, and mesolimbic structures, and propose hypotheses about how stimulation delivered at specific learning phases could strengthen threat extinction.

Neural mass model-based study of frontal-temporal theta oscillations in human subjects during the performance of a cognitive control task

Cognitive control, the ability to rapidly shift one's attention and behavioral strategy in response to environmental changes, is often compromised across psychiatric disorders. One of the well-validated behavioral paradigms for tapping into the cognitive control circuits is a cognitive interference task, where subjects must suppress a natural response to follow a less intuitive rule. Slower response times on these tasks indicate difficulty exerting control to overcome response conflict. Conflict evokes robust electrophysiological signatures, such as theta (4-8 Hz) oscillations in the prefrontal cortex (PFC). However, the underlying neural mechanisms of conflict-evoked theta oscillations in the PFC are not clear. The objective of this work is to use a neural mass model (NMM) to find feasible cortical networks generating theta oscillations during conflict processing in human subjects. We used intracranial EEG (iEEG) recorded from dorsolateral PFC (dIPFC) and lateral temporal lobe (LTL) of human subjects with intractable epilepsy undergoing invasive monitoring, while they performed a multi-source interference task (MSIT). We used a dynamic causal modeling (DCM) framework to simulate dIPFC-LTL theta using a Jansen-Rit NMM. We found significant evidence for an LTL input into the dlPFC during the initial 500 ms of conflict processing compared to a bidirectional connection between the dlPFC and LTL. We conclude that a neural mass modeling framework can be used to elucidate candidate mechanisms of neural oscillations underlying conflict resolution in human subjects. Clinical Relevance- This can be used to find feasible target mechanisms for designing therapy in patients with compromised cognitive control. This framework can also be expanded to serve as an in-silico test bed for designing and testing neuromodulatory interventions such as electrical stimulation for improving cognitive control in mood/anxiety disorders.

The Nencki-Symfonia electroencephalography/event-related potential dataset: Multiple cognitive tasks and resting-state data collected in a sample of healthy adults

Background

One of the goals of neuropsychology is to understand the brain mechanisms underlying aspects of attention and cognitive control. Several tasks have been developed as a part of this body of research, however their results are not always consistent. A reliable comparison of the data and a synthesis of study conclusions has been precluded by multiple methodological differences. Here, we describe a publicly available, high-density electroencephalography (EEG) dataset obtained from 42 healthy young adults while they performed 3 cognitive tasks: (i) an extended multi-source interference task; (ii) a 3-stimuli oddball task; (iii) a control, simple reaction task; and (iv) a resting-state protocol. Demographic and psychometric information are included within the dataset.

Dataset Validation

First, data validation confirmed acceptable quality of the obtained EEG signals. Typical event-related potential (ERP) waveforms were obtained, as expected for attention and cognitive control tasks (i.e., N200, P300, N450). Behavioral results showed the expected progression of reaction times and error rates, which confirmed the effectiveness of the applied paradigms.

Conclusions

This dataset is well suited for neuropsychological research regarding common and distinct mechanisms involved in different cognitive tasks. Using this dataset, researchers can compare a wide range of classical EEG/ERP features across tasks for any selected subset of electrodes. At the same time, 128-channel EEG recording allows for source localization and detailed connectivity studies. Neurophysiological measures can be correlated with additional psychometric data obtained from the same participants. This dataset can also be used to develop and verify novel analytical and classification approaches that can advance the field of deep/machine learning algorithms, recognition of single-trial ERP responses to different task conditions, and detection of EEG/ERP features for use in brain-computer interface applications.

Long-term ecological assessment of intracranial electrophysiology synchronized to behavioral markers in obsessive-compulsive disorder

Detection of neural signatures related to pathological behavioral states could enable adaptive deep brain stimulation (DBS), a potential strategy for improving efficacy of DBS for neurological and psychiatric disorders. This approach requires identifying neural biomarkers of relevant behavioral states, a task best performed in ecologically valid environments. Here, in human participants with obsessive-compulsive disorder (OCD) implanted with recording-capable DBS devices, we synchronized chronic ventral striatum local field potentials with relevant, disease-specific behaviors. We captured over 1,000 h of local field potentials in the clinic and at home during unstructured activity, as well as during DBS and exposure therapy. The wide range of symptom severity over which the data were captured allowed us to identify candidate neural biomarkers of OCD symptom intensity. This work demonstrates the feasibility and utility of capturing chronic intracranial electrophysiology during daily symptom fluctuations to enable neural biomarker identification, a prerequisite for future development of adaptive DBS for OCD and other psychiatric disorders.

Spectral Features Based Decoding of Task Engagement: The Role of Theta and High Gamma Bands in Cognitive Control

This paper analyzes local field potentials (LFP) from 10 human subjects to discover frequency-dependent biomarkers of cognitive conflict. We utilize cortical and sub-cortical LFP recordings from the subjects during a cognitive task known as the Multi-Source Interference Task (MSIT). We decode the task engagement and discover biomarkers that may facilitate closed-loop neuromodulation to enhance cognitive control. First, we show that spectral power features in predefined frequency bands can be used to classify task and non-task segments with a median accuracy of 88.1%. Here the features are first ranked using the Bayes Factor and then used as inputs to subject-specific linear support vector machine classifiers. Second, we show that theta (4-8 Hz) band, and high gamma (65-200 Hz) band oscillations are modulated during the task performance. Third, by isolating time-series from specific brain regions of interest, we observe that a subset of the dorsolateral prefrontal cortex features is sufficient to decode the task states. The paper shows that cognitive control evokes robust neurological signatures, especially in the prefrontal cortex (PFC).

Decoding Human Cognitive Control Using Functional Connectivity of Local Field Potentials

Many patients with mental illnesses characterized by impaired cognitive control have no relief from gold-standard clinical treatments resulting in a pressing need for new alternatives. This paper develops a neural decoder to detect task engagement in ten human subjects during a conflict-based behavioral task known as the multi-source interference task (MSIT). Task engagement is of particular interest here because closed-loop brain stimulation during those states can augment decision-making. The functional connectivity patterns of the electrodes are extracted. A principal component analysis of these patterns is carried out and the ranked principal components are used as inputs to train subject-specific linear support vector machine classifiers. In this paper, we show that task engagement can be differentiated from background brain activity with a median accuracy of 89.7%. This was accomplished by constructing distributed functional networks from local field potentials recording during the task performance. A further challenge is that goal-directed efforts take place over higher temporal resolution. Task engagement must thus be detected at a similar rate for proactive intervention. We show that our algorithms can detect task engagement from neural recordings in less than 2 seconds; this can be further improved using an application-specific device.

Closed-Loop Neural Prostheses With On-Chip Intelligence: A Review and a Low-Latency Machine Learning Model for Brain State Detection

The application of closed-loop approaches in systems neuroscience and therapeutic stimulation holds great promise for revolutionizing our understanding of the brain and for developing novel neuromodulation therapies to restore lost functions. Neural prostheses capable of multi-channel neural recording, on-site signal processing, rapid symptom detection, and closed-loop stimulation are critical to enabling such novel treatments. However, the existing closed-loop neuromodulation devices are too simplistic and lack sufficient on-chip processing and intelligence. In this paper, we first discuss both commercial and investigational closed-loop neuromodulation devices for brain disorders. Next, we review state-of-the-art neural prostheses with on-chip machine learning, focusing on application-specific integrated circuits (ASIC). System requirements, performance and hardware comparisons, design trade-offs, and hardware optimization techniques are discussed. To facilitate a fair comparison and guide design choices among various on-chip classifiers, we propose a new energy-area (E-A) efficiency figure of merit that evaluates hardware efficiency and multi-channel scalability. Finally, we present several techniques to improve the key design metrics of tree-based on-chip classifiers, both in the context of ensemble methods and oblique structures. A novel Depth-Variant Tree Ensemble (DVTE) is proposed to reduce processing latency (e.g., by 2.5× on seizure detection task). We further develop a cost-aware learning approach to jointly optimize the power and latency metrics. We show that algorithm-hardware co-design enables the energy- and memory-optimized design of tree-based models, while preserving a high accuracy and low latency. Furthermore, we show that our proposed tree-based models feature a highly interpretable decision process that is essential for safety-critical applications such as closed-loop stimulation.

Region-Level Functional and Effective Network Analysis of Human Brain During Cognitive Task Engagement

Mental disorders are a major source of disability, with few effective treatments. It has recently been argued that these diseases might be effectively treated by focusing on decision-making, and specifically remediating decision-making deficits that act as "ingredients" in these disorders. Prior work showed that direct electrical brain stimulation can enhance human cognitive control, and consequently decision-making. This raises a challenge of detecting cognitive control lapses directly from electrical brain activity. Here, we demonstrate approaches to overcome that challenge. We propose a novel method, referred to as maximal variance node merging (MVNM), that merges nodes within a brain region to construct informative inter-region brain networks. We employ this method to estimate functional (correlational) and effective (causal) networks using local field potentials (LFP) during a cognitive behavioral task. The effective networks computed using convergent cross mapping differentiate task engagement from background neural activity with 85% median classification accuracy. We also derive task engagement networks (TENs): networks that constitute the most discriminative inter-region connections. Subsequent graph analysis illustrates the crucial role of the dorsolateral prefrontal cortex (dlPFC) in task engagement, consistent with a widely accepted model for cognition. We also show that task engagement is linked to prefrontal cortex theta (4-8 Hz) oscillations. We, therefore, identify objective biomarkers associated with task engagement. These approaches may generalize to other cognitive functions, forming the basis of a network-based approach to detecting and rectifying decision deficits.

Mental workload classification based on ignored auditory probes and spatial covariance

Objective.Estimation of mental workload (MWL) levels by electroencephalography (EEG)-based mental state monitoring systems has been widely explored. Using event-related potentials (ERPs), elicited by ignored auditory probes, minimizes intrusiveness and has shown high performance for estimating MWL level when tested in laboratory situations. However, when facing real-world applications, the characteristics of ERP waveforms, like latency and amplitude, are often affected by noise, which leads to a decrease in classification performance. One approach to mitigating this is using spatial covariance, which is less sensitive to latency and amplitude distortion. In this study, we used ignored auditory probes in a single-stimulus paradigm and tested Riemannian processed covariance-based features for MWL level estimation in a realistic flight-control task.Approach.We recorded EEG data with an eight-channel system from participants while they performed a simulated drone-control task and manipulated MWL levels (high and low) by task difficulty. We compared support vector machine classification performance based on frequency band power features versus features generated via the Riemannian log map operator from spatial covariance matrices. We also compared accuracy of using data segmented as auditory ERPs versus non-ERPs, for which data windows did not overlap with the ERPs.Main results.Classification accuracy of both types of features showed no significant difference between ERPs and non-ERPs. When we ignore auditory stimuli to perform continuous decoding, covariance-based features in the gamma band had area under the receiver-operating-characteristic curve (AUC) of 0.883, which was significantly higher than band power features (AUC = 0.749).Significance.This study demonstrates that Riemannian-processed covariance features are viable for MWL classification under a realistic experimental scenario.

Predicting task performance from biomarkers of mental fatigue in global brain activity

Objective.

Detection and early prediction of mental fatigue (i.e. shifts in vigilance), could be used to adapt neuromodulation strategies to effectively treat patients suffering from brain injury and other indications with prominent chronic mental fatigue.

Approach.

In this study, we analyzed electrocorticography (ECoG) signals chronically recorded from two healthy non-human primates (NHP) as they performed a sustained attention task over extended periods of time. We employed a set of spectrotemporal and connectivity biomarkers of the ECoG signals to identify periods of mental fatigue and a gradient boosting classifier to predict performance, up to several seconds prior to the behavioral response.

Main results.

Wavelet entropy and the instantaneous amplitude and frequency were among the best single features across sessions in both NHPs. The classification performance using higher order spectral-temporal (HOST) features was significantly higher than that of conventional spectral power features in both NHPs. Across the 99 sessions analyzed, average F1 scores of 77.5%±8.2% and 91.2%±3.6%, and accuracy of 79.5%±8.9% and 87.6%±3.9 % for the classifier were obtained for each animal, respectively.

Significance.

Our results here demonstrate the feasibility of predicting performance and detecting periods of mental fatigue by analyzing ECoG signals, and that this general approach, in principle, could be used for closed-loop control of neuromodulation strategies.

Decoding working memory task condition using magnetoencephalography source level long-range phase coupling patterns

OBJECTIVE

The objective of the study is to identify phase coupling patterns that are shared across subjects via a machine learning approach that utilises source space magnetoencephalography (MEG) phase coupling data from a working memory (WM) task. Indeed, phase coupling of neural oscillations is putatively a key factor for communication between distant brain areas and is therefore crucial in performing cognitive tasks, including WM. Previous studies investigating phase coupling during cognitive tasks have often focused on a few a priori selected brain areas or a specific frequency band, and the need for data-driven approaches has been recognised. Machine learning techniques have emerged as valuable tools for the analysis of neuroimaging data since they catch fine-grained differences in the multivariate signal distribution. Here, we expect that these techniques applied to MEG phase couplings can reveal WM-related processes that are shared across individuals.

APPROACH

We analysed WM data collected as part of the Human Connectome Project. The MEG data were collected while subjects (n = 83) performed N-back WM tasks in two different conditions, namely 2-back (WM condition) and 0-back (control condition). We estimated phase coupling patterns (multivariate phase slope index) for both conditions and for theta, alpha, beta, and gamma bands. The obtained phase coupling data were then used to train a linear support vector machine in order to classify which task condition the subject was performing with an across-subject cross-validation approach. The classification was performed separately based on the data from individual frequency bands and with all bands combined (multiband). Finally, we evaluated the relative importance of the different features (phase couplings) for classification by the means of feature selection probability.

MAIN RESULTS

The WM condition and control condition were successfully classified based on the phase coupling patterns in the theta (62% accuracy) and alpha bands (60% accuracy) separately. Importantly, the multiband classification showed that phase coupling patterns not only in the theta and alpha but also in the gamma bands are related to WM processing, as testified by improvement in classification performance (71%).

SIGNIFICANCE

Our study successfully decoded WM tasks using MEG source space functional connectivity. Our approach, combining across-subject classification and a multidimensional metric recently developed by our group, is able to detect patterns of connectivity that are shared across individuals. In other words, the results are generalisable to new individuals and allow meaningful interpretation of task-relevant phase coupling patterns.

Deep brain stimulation for psychiatric disorders: From focal brain targets to cognitive networks

Deep brain stimulation (DBS) is a promising intervention for treatment-resistant psychiatric disorders, particularly major depressive disorder (MDD) and obsessive-compulsive disorder (OCD). Up to 90% of patients who have not recovered with therapy or medication have reported benefit from DBS in open-label studies. Response rates in randomized controlled trials (RCTs), however, have been much lower. This has been argued to arise from surgical variability between sites, and recent psychiatric DBS research has focused on refining targeting through personalized imaging. Much less attention has been given to the fact that psychiatric disorders arise from dysfunction in distributed brain networks, and that DBS likely acts by altering communication within those networks. This is in part because psychiatric DBS research relies on subjective rating scales that make it difficult to identify network biomarkers. Here, we overview recent DBS RCT results in OCD and MDD, as well as the follow-on imaging studies. We present evidence for a new approach to studying DBS' mechanisms of action, focused on measuring objective cognitive/emotional deficits that underpin these and many other mental disorders. Further, we suggest that a focus on cognition could lead to reliable network biomarkers at an electrophysiologic level, especially those related to inter-regional synchrony of the local field potential (LFP). Developing the network neuroscience of DBS has the potential to finally unlock the potential of this highly specific therapy.

Patterns of Focal- and Large-Scale Synchronization in Cognitive Control and Inhibition: A Review

Neural synchronization patterns are involved in several complex cognitive functions and constitute a growing trend in neuroscience research. While synchrony patterns in working memory have been extensively discussed, a complete understanding of their role in cognitive control and inhibition is still elusive. Here, we provide an up-to-date review on synchronization patterns underlying behavioral inhibition, extrapolating common grounds, and dissociating features with other inhibitory functions. Moreover, we suggest a schematic conceptual framework and highlight existing gaps in the literature, current methodological challenges, and compelling research questions for future studies.

Prefrontal cortex and cognitive control: new insights from human electrophysiology

Cognitive control, the ability to regulate one's cognition and actions on the basis of over-riding goals, is impaired in many psychiatric conditions. Although control requires the coordinated function of several prefrontal cortical regions, it has been challenging to determine how they work together, in part because doing so requires simultaneous recordings from multiple regions. Here, we provide a précis of cognitive control and describe the beneficial consequences of recent advances in neurosurgical practice that make large-scale prefrontal cortical network recordings possible in humans. Such recordings implicate inter-regional theta (5-8 Hz) local field potential (LFP) synchrony as a key element in cognitive control. Major open questions include how theta might influence other oscillations within these networks, the precise timing of information flow between these regions, and how perturbations such as brain stimulation might demonstrate the causal role of LFP phenomena. We propose that an increased focus on human electrophysiology is essential for an understanding of the neural basis of cognitive control.