Mental Workload Classification and Tasks Detection in Multitasking: Deep Learning Insights from EEG Study

While the term task load (TL) refers to external task demands, the amount of work, or the number of tasks to be performed, mental workload (MWL) refers to the individual's effort, mental capacity, or cognitive resources utilized while performing a task. MWL in multitasking scenarios is often closely linked with the quantity of tasks a person is handling within a given timeframe. In this study, we challenge this hypothesis from the perspective of electroencephalography (EEG) using a deep learning approach. We conducted an EEG experiment with 50 participants performing NASA Multi-Attribute Task Battery II (MATB-II) under 4 different task load levels. We designed a convolutional neural network (CNN) to help with two distinct classification tasks. In one setting, the CNN was used to classify EEG segments based on their task load level. In another setting, the same CNN architecture was trained again to detect the presence of individual MATB-II subtasks. Results show that, while the model successfully learns to detect whether a particular subtask is active in a given segment (i.e., to differentiate between different subtasks-related EEG patterns), it struggles to differentiate between the two highest levels of task load (i.e., to distinguish MWL-related EEG patterns). We speculate that the challenge comes from two factors: first, the experiment was designed in a way that these two highest levels differed only in the quantity of work within a given timeframe; and second, the participants' effective adaptation to increased task demands, as evidenced by low error rates. Consequently, this indicates that under such conditions in multitasking, EEG may not reflect distinct enough patterns to differentiate higher levels of task load.

Introducing a New Mobile Electroencephalography System and Evaluating Its Quality in Comparison to Clinical Electroencephalography

Electroencephalography (EEG) is a crucial tool in cognitive neuroscience, enabling the study of neurophysiological function by measuring the brain's electrical activity. Its applications include perception, learning, memory, language, decision making and neural network mapping. Recently, interest has surged in extending EEG measurements to domestic environments. However, the high costs associated with traditional laboratory EEG systems have hindered accessibility for many individuals and researchers in education, research, and medicine. To tackle this, a mobile-EEG device named "DreamMachine" was developed. A more affordable alternative to both lab-based EEG systems and existing mobile-EEG devices. This system boasts 24 channels, 24-bit resolution, up to 6 h of battery life, portability, and a low price. Our open-source and open-hardware approach empowers cognitive neuroscience, especially in education, learning, and research, opening doors to more accessibility. This paper introduces the DreamMachine's design and compares it with the lab-based EEG system "asalabTM" in an eyes-open and eyes-closed experiment. The Alpha band exhibited higher power in the power spectrum during eyes-closed conditions, whereas the eyes-open condition showed increased power specifically within the Delta frequency range. Our analysis confirms that the DreamMachine accurately records brain activity, meeting the necessary standards when compared to the asalabTM system.

Frontal-midline theta and posterior alpha oscillations index early processing of spatial representations during active navigation

Previous research has demonstrated that humans combine multiple sources of spatial information such as self-motion and landmark cues, while navigating through an environment. However, it is unclear whether this involves comparing multiple representations obtained from different sources during navigation (parallel hypothesis) or building a representation first based on self-motion cues and then combining with landmarks later (serial hypothesis). We tested these two hypotheses (parallel vs. serial) in an active navigation task using wireless mobile scalp EEG recordings. Participants walked through an immersive virtual hallway with or without conflicts between self-motion and landmarks (i.e., intersections) and pointed toward the starting position of the hallway. We employed the oscillatory signals recorded during mobile wireless scalp EEG as means of identifying when participant representations based on self-motion vs. landmark cues might have first emerged. We found that path segments, including intersections present early during navigation, were more strongly associated with later pointing error, regardless of when they appeared during encoding. We also found that there was sufficient information contained within the frontal-midline theta and posterior alpha oscillatory signals in the earliest segments of navigation involving intersections to decode condition (i.e., conflicting vs. not conflicting). Together, these findings suggest that intersections play a pivotal role in the early development of spatial representations, suggesting that memory representations for the geometry of walked paths likely develop early during navigation, in support of the parallel hypothesis.

Context effects on object recognition in real-world environments: A study protocol

Background: The environments that we live in impact on our ability to recognise objects, with recognition being facilitated when objects appear in expected locations (congruent) compared to unexpected locations (incongruent). However, these findings are based on experiments where the object is isolated from its environment. Moreover, it is not clear which components of the recognition process are impacted by the environment. In this experiment, we seek to examine the impact real world environments have on object recognition. Specifically, we will use mobile electroencephalography (mEEG) and augmented reality (AR) to investigate how the visual and semantic processing aspects of object recognition are changed by the environment. Methods: We will use AR to place congruent and incongruent virtual objects around indoor and outdoor environments. During the experiment a total of 34 participants will walk around the environments and find these objects while we record their eye movements and neural signals. We will perform two primary analyses. First, we will analyse the event-related potential (ERP) data using paired samples t-tests in the N300/400 time windows in an attempt to replicate congruency effects on the N300/400. Second, we will use representational similarity analysis (RSA) and computational models of vision and semantics to determine how visual and semantic processes are changed by congruency. Conclusions: Based on previous literature, we hypothesise that scene-object congruence would facilitate object recognition. For ERPs, we predict a congruency effect in the N300/N400, and for RSA we predict that higher level visual and semantic information will be represented earlier for congruent scenes than incongruent scenes. By collecting mEEG data while participants are exploring a real-world environment, we will be able to determine the impact of a natural context on object recognition, and the different processing stages of object recognition.

Design and Validation of a Low-Cost Mobile EEG-Based Brain-Computer Interface

Objective: We designed and validated a wireless, low-cost, easy-to-use, mobile, dry-electrode headset for scalp electroencephalography (EEG) recordings for closed-loop brain-computer (BCI) interface and internet-of-things (IoT) applications. Approach: The EEG-based BCI headset was designed from commercial off-the-shelf (COTS) components using a multi-pronged approach that balanced interoperability, cost, portability, usability, form factor, reliability, and closed-loop operation. Main Results: The adjustable headset was designed to accommodate 90% of the population. A patent-pending self-positioning dry electrode bracket allowed for vertical self-positioning while parting the user's hair to ensure contact of the electrode with the scalp. In the current prototype, five EEG electrodes were incorporated in the electrode bracket spanning the sensorimotor cortices bilaterally, and three skin sensors were included to measure eye movement and blinks. An inertial measurement unit (IMU) provides monitoring of head movements. The EEG amplifier operates with 24-bit resolution up to 500 Hz sampling frequency and can communicate with other devices using 802.11 b/g/n WiFi. It has high signal-to-noise ratio (SNR) and common-mode rejection ratio (CMRR) (121 dB and 110 dB, respectively) and low input noise. In closed-loop BCI mode, the system can operate at 40 Hz, including real-time adaptive noise cancellation and 512 MB of processor memory. It supports LabVIEW as a backend coding language and JavaScript (JS), Cascading Style Sheets (CSS), and HyperText Markup Language (HTML) as front-end coding languages and includes training and optimization of support vector machine (SVM) neural classifiers. Extensive bench testing supports the technical specifications and human-subject pilot testing of a closed-loop BCI application to support upper-limb rehabilitation and provides proof-of-concept validation for the device's use at both the clinic and at home. Significance: The usability, interoperability, portability, reliability, and programmability of the proposed wireless closed-loop BCI system provides a low-cost solution for BCI and neurorehabilitation research and IoT applications.

Investigating the attentional focus to workplace-related soundscapes in a complex audio-visual-motor task using EEG

Introduction

In demanding work situations (e.g., during a surgery), the processing of complex soundscapes varies over time and can be a burden for medical personnel. Here we study, using mobile electroencephalography (EEG), how humans process workplace-related soundscapes while performing a complex audio-visual-motor task (3D Tetris). Specifically, we wanted to know how the attentional focus changes the processing of the soundscape as a whole.

Method

Participants played a game of 3D Tetris in which they had to use both hands to control falling blocks. At the same time, participants listened to a complex soundscape, similar to what is found in an operating room (i.e., the sound of machinery, people talking in the background, alarm sounds, and instructions). In this within-subject design, participants had to react to instructions (e.g., "place the next block in the upper left corner") and to sounds depending on the experimental condition, either to a specific alarm sound originating from a fixed location or to a beep sound that originated from varying locations. Attention to the alarm reflected a narrow attentional focus, as it was easy to detect and most of the soundscape could be ignored. Attention to the beep reflected a wide attentional focus, as it required the participants to monitor multiple different sound streams.

Results and discussion

Results show the robustness of the N1 and P3 event related potential response during this dynamic task with a complex auditory soundscape. Furthermore, we used temporal response functions to study auditory processing to the whole soundscape. This work is a step toward studying workplace-related sound processing in the operating room using mobile EEG.

Neuroergonomics on the Go: An Evaluation of the Potential of Mobile EEG for Workplace Assessment and Design

OBJECTIVE

We demonstrate and discuss the use of mobile electroencephalogram (EEG) for neuroergonomics. Both technical state of the art as well as measures and cognitive concepts are systematically addressed.

BACKGROUND

Modern work is increasingly characterized by information processing. Therefore, the examination of mental states, mental load, or cognitive processing during work is becoming increasingly important for ergonomics.

RESULTS

Mobile EEG allows to measure mental states and processes under real live conditions. It can be used for various research questions in cognitive neuroergonomics. Besides measures in the frequency domain that have a long tradition in the investigation of mental fatigue, task load, and task engagement, new approaches-like blink-evoked potentials-render event-related analyses of the EEG possible also during unrestricted behavior.

CONCLUSION

Mobile EEG has become a valuable tool for evaluating mental states and mental processes on a highly objective level during work. The main advantage of this technique is that working environments don't have to be changed while systematically measuring brain functions at work. Moreover, the workflow is unaffected by such neuroergonomic approaches.

Neurological Outpatients Prefer EEG Home-Monitoring over Inpatient Monitoring-An Analysis Based on the UTAUT Model

Home monitoring examinations offer diagnostic and economic advantages compared to inpatient monitoring. In addition, these technical solutions support the preservation of health care in rural areas in the absence of local care providers. The acceptance of patients is crucial for the implementation of home monitoring concepts. The present research assesses the preference for a health service that is to be introduced, namely an EEG home-monitoring of neurological outpatients-using a mobile, dry-electrode EEG (electroencephalography) system-in comparison to the traditional long-time EEG examination in a hospital. Results of a representative study for Germany (n = 421) reveal a preference for home monitoring. Importantly, this preference is partially driven by a video explaining the home monitoring system. We subsequently analyzed factors that influence the behavioral intention (BI) to use the new EEG system, drawing on an extended Unified Theory of Acceptance and Use of Technology (UTAUT) model. The strongest positive predictor of BI is the belief that EEG home-monitoring will improve health quality, while computer anxiety and effort expectancy represent the strongest barriers. Furthermore, we find the UTAUT model's behavioral intention construct to predict the patients' decision for or against home monitoring more strongly than any other patient's characteristic such as gender, health condition, or age, underlying the model's usefulness.

Mobile electroencephalography captures differences of walking over even and uneven terrain but not of single and dual-task gait

Walking on natural terrain while performing a dual-task, such as typing on a smartphone is a common behavior. Since dual-tasking and terrain change gait characteristics, it is of interest to understand how altered gait is reflected by changes in gait-associated neural signatures. A study was performed with 64-channel electroencephalography (EEG) of healthy volunteers, which was recorded while they walked over uneven and even terrain outdoors with and without performing a concurrent task (self-paced button pressing with both thumbs). Data from n = 19 participants (M = 24 years, 13 females) were analyzed regarding gait-phase related power modulations (GPM) and gait performance (stride time and stride time-variability). GPMs changed significantly with terrain, but not with the task. Descriptively, a greater beta power decrease following right-heel strikes was observed on uneven compared to even terrain. No evidence of an interaction was observed. Beta band power reduction following the initial contact of the right foot was more pronounced on uneven than on even terrain. Stride times were longer on uneven compared to even terrain and during dual- compared to single-task gait, but no significant interaction was observed. Stride time variability increased on uneven terrain compared to even terrain but not during single- compared to dual-tasking. The results reflect that as the terrain difficulty increases, the strides become slower and more irregular, whereas a secondary task slows stride duration only. Mobile EEG captures GPM differences linked to terrain changes, suggesting that the altered gait control demands and associated cortical processes can be identified. This and further studies may help to lay the foundation for protocols assessing the cognitive demand of natural gait on the motor system.

Ear-EEG compares well to cap-EEG in recording auditory ERPs: a quantification of signal loss

Objective:Ear-EEG (Electroencephalography) allows to record brain activity using only a few electrodes located close to the ear. Ear-EEG is comfortable and easy to apply, facilitating beyond-the-lab EEG recordings in everyday life. With the unobtrusive setup, a person wearing it can blend in, allowing unhindered EEG recordings in social situations. However, compared to classical cap-EEG, only a small part of the head is covered with electrodes. Most scalp positions that are known from established EEG research are not covered by ear-EEG electrodes, making the comparison between the two approaches difficult and might hinder the transition from cap-based lab studies to ear-based beyond-the-lab studies.Approach:We here provide a reference data-set comparing ear-EEG and cap-EEG directly for four different auditory event-related potentials (ERPs): N100, MMN, P300 and N400. We show how the ERPs are reflected when using only electrodes around the ears.Main results:We find that significant condition differences for all ERP-components could be recorded using only ear-electrodes. The effect sizes were moderate to high on the single subject level. Morphology and temporal evolution of signals recorded from around-the-ear resemble highly those from standard scalp-EEG positions. We found a reduction in effect size (signal loss) for the ear-EEG electrodes compared to cap-EEG of 21-44%. The amount of signal loss depended on the ERP-component; we observed the lowest percentage signal loss for the N400 and the highest percentage signal loss for the N100. Our analysis further shows that no single channel position around the ear is optimal for recording all ERP-components or all participants, speaking in favor of multi-channel ear-EEG solutions.Significance:Our study provides reference results for future studies employing ear-EEG.

Robust learning from corrupted EEG with dynamic spatial filtering

Building machine learning models using EEG recorded outside of the laboratory setting requires methods robust to noisy data and randomly missing channels. This need is particularly great when working with sparse EEG montages (1-6 channels), often encountered in consumer-grade or mobile EEG devices. Neither classical machine learning models nor deep neural networks trained end-to-end on EEG are typically designed or tested for robustness to corruption, and especially to randomly missing channels. While some studies have proposed strategies for using data with missing channels, these approaches are not practical when sparse montages are used and computing power is limited (e.g., wearables, cell phones). To tackle this problem, we propose dynamic spatial filtering (DSF), a multi-head attention module that can be plugged in before the first layer of a neural network to handle missing EEG channels by learning to focus on good channels and to ignore bad ones. We tested DSF on public EEG data encompassing ∼4,000 recordings with simulated channel corruption and on a private dataset of ∼100 at-home recordings of mobile EEG with natural corruption. Our proposed approach achieves the same performance as baseline models when no noise is applied, but outperforms baselines by as much as 29.4% accuracy when significant channel corruption is present. Moreover, DSF outputs are interpretable, making it possible to monitor the effective channel importance in real-time. This approach has the potential to enable the analysis of EEG in challenging settings where channel corruption hampers the reading of brain signals.

Real-Time Audio Processing of Real-Life Soundscapes for EEG Analysis: ERPs Based on Natural Sound Onsets

With smartphone-based mobile electroencephalography (EEG), we can investigate sound perception beyond the lab. To understand sound perception in the real world, we need to relate naturally occurring sounds to EEG data. For this, EEG and audio information need to be synchronized precisely, only then it is possible to capture fast and transient evoked neural responses and relate them to individual sounds. We have developed Android applications (AFEx and Record-a) that allow for the concurrent acquisition of EEG data and audio features, i.e., sound onsets, average signal power (RMS), and power spectral density (PSD) on smartphone. In this paper, we evaluate these apps by computing event-related potentials (ERPs) evoked by everyday sounds. One participant listened to piano notes (played live by a pianist) and to a home-office soundscape. Timing tests showed a stable lag and a small jitter (< 3 ms) indicating a high temporal precision of the system. We calculated ERPs to sound onsets and observed the typical P1-N1-P2 complex of auditory processing. Furthermore, we show how to relate information on loudness (RMS) and spectra (PSD) to brain activity. In future studies, we can use this system to study sound processing in everyday life.

Benchmarking cEEGrid and Solid Gel-Based Electrodes to Classify Inattentional Deafness in a Flight Simulator

Transfer from experiments in the laboratory to real-life tasks is challenging due notably to the inability to reproduce the complexity of multitasking dynamic everyday life situations in a standardized lab condition and to the bulkiness and invasiveness of recording systems preventing participants from moving freely and disturbing the environment. In this study, we used a motion flight simulator to induce inattentional deafness to auditory alarms, a cognitive difficulty arising in complex environments. In addition, we assessed the possibility of two low-density EEG systems a solid gel-based electrode Enobio (Neuroelectrics, Barcelona, Spain) and a gel-based cEEGrid (TMSi, Oldenzaal, Netherlands) to record and classify brain activity associated with inattentional deafness (misses vs. hits to odd sounds) with a small pool of expert participants. In addition to inducing inattentional deafness (missing auditory alarms) at much higher rates than with usual lab tasks (34.7% compared to the usual 5%), we observed typical inattentional deafness-related activity in the time domain but also in the frequency and time-frequency domains with both systems. Finally, a classifier based on Riemannian Geometry principles allowed us to obtain more than 70% of single-trial classification accuracy for both mobile EEG, and up to 71.5% for the cEEGrid (TMSi, Oldenzaal, Netherlands). These results open promising avenues toward detecting cognitive failures in real-life situations, such as real flight.

Performance of Four Commercial Wearable Sleep-Tracking Devices Tested Under Unrestricted Conditions at Home in Healthy Young Adults

PURPOSE

Commercial wearable sleep-tracking devices are growing in popularity and in recent studies have performed well against gold standard sleep measurement techniques. However, most studies were conducted in controlled laboratory conditions. We therefore aimed to test the performance of devices under naturalistic unrestricted home sleep conditions.

PARTICIPANTS AND METHODS

Healthy young adults (n = 21; 12 women, 9 men; 29.0 ± 5.0 years, mean ± SD) slept at home under unrestricted conditions for 1 week using a set of commercial wearable sleep-tracking devices and completed daily sleep diaries. Devices included the Fatigue Science Readiband, Fitbit Inspire HR, Oura ring, and Polar Vantage V Titan. Participants also wore a research-grade actigraphy watch (Philips Respironics Actiwatch 2) for comparison. To assess performance, all devices were compared with a high performing mobile sleep electroencephalography headband device (Dreem 2). Analyses included epoch-by-epoch and sleep summary agreement comparisons.

RESULTS

Devices accurately tracked sleep-wake summary metrics (ie, time in bed, total sleep time, sleep efficiency, sleep latency, wake after sleep onset) on most nights but performed best on nights with higher sleep efficiency. Epoch-by-epoch sensitivity (for sleep) and specificity (for wake), respectively, were as follows: Actiwatch (0.95, 0.35), Fatigue Science (0.94, 0.40), Fitbit (0.93, 0.45), Oura (0.94, 0.41), and Polar (0.96, 0.35). Sleep stage-tracking performance was mixed, with high variability.

CONCLUSION

As in previous studies, all devices were better at detecting sleep than wake, and most devices compared favorably to actigraphy in wake detection. Devices performed best on nights with more consolidated sleep patterns. Unrestricted sleep TIB differences were accurately tracked on most nights. High variability in sleep stage-tracking performance suggests that these devices, in their current form, are still best utilized for tracking sleep-wake outcomes and not sleep stages. Most commercial wearables exhibited promising performance for tracking sleep-wake in real-world conditions, further supporting their consideration as an alternative to actigraphy.

Neural tracking to go: auditory attention decoding and saliency detection with mobile EEG

OBJECTIVE

Neuro-steered assistive technologies have been suggested to offer a major advancement in future devices like neuro-steered hearing aids. Auditory attention decoding methods would in that case allow for identification of an attended speaker within complex auditory environments, exclusively from neural data. Decoding the attended speaker using neural information has so far only been done in controlled laboratory settings. Yet, it is known that ever-present factors like distraction and movement are reflected in the neural signal parameters related to attention.

APPROACH

Thus, in the current study we applied a two-competing speaker paradigm to investigate performance of a commonly applied EEG-based auditory attention decoding (AAD) model outside of the laboratory during leisure walking and distraction. Unique environmental sounds were added to the auditory scene and served as distractor events.

MAIN RESULTS

The current study shows, for the first time, that the attended speaker can be accurately decoded during natural movement. At a temporal resolution of as short as 5-seconds and without artifact attenuation, decoding was found to be significantly above chance level. Further, as hypothesized, we found a decrease in attention to the to-be-attended and the to-be-ignored speech stream after the occurrence of a salient event. Additionally, we demonstrate that it is possible to predict neural correlates of distraction with a computational model of auditory saliency based on acoustic features.

CONCLUSION

Taken together, our study shows that auditory attention tracking outside of the laboratory in ecologically valid conditions is feasible and a step towards the development of future neural-steered hearing aids.

A high-density 256-channel cap for dry electroencephalography

High‐density electroencephalography (HD‐EEG) is currently limited to laboratory environments since state‐of‐the‐art electrode caps require skilled staff and extensive preparation. We propose and evaluate a 256‐channel cap with dry multipin electrodes for HD‐EEG. We describe the designs of the dry electrodes made from polyurethane and coated with Ag/AgCl. We compare in a study with 30 volunteers the novel dry HD‐EEG cap to a conventional gel‐based cap for electrode‐skin impedances, resting state EEG, and visual evoked potentials (VEP). We perform wearing tests with eight electrodes mimicking cap applications on real human and artificial skin. Average impedances below 900 kΩ for 252 out of 256 dry electrodes enables recording with state‐of‐the‐art EEG amplifiers. For the dry EEG cap, we obtained a channel reliability of 84% and a reduction of the preparation time of 69%. After exclusion of an average of 16% (dry) and 3% (gel‐based) bad channels, resting state EEG, alpha activity, and pattern reversal VEP can be recorded with less than 5% significant differences in all compared signal characteristics metrics. Volunteers reported wearing comfort of 3.6 ± 1.5 and 4.0 ± 1.8 for the dry and 2.5 ± 1.0 and 3.0 ± 1.1 for the gel‐based cap prior and after the EEG recordings, respectively (scale 1–10). Wearing tests indicated that up to 3,200 applications are possible for the dry electrodes. The 256‐channel HD‐EEG dry electrode cap overcomes the principal limitations of HD‐EEG regarding preparation complexity and allows rapid application by not medically trained persons, enabling new use cases for HD‐EEG.

Mobile Electroencephalography for Studying Neural Control of Human Locomotion

Walking or running in real-world environments requires dynamic multisensory processing within the brain. Studying supraspinal neural pathways during human locomotion provides opportunities to better understand complex neural circuity that may become compromised due to aging, neurological disorder, or disease. Knowledge gained from studies examining human electrical brain dynamics during gait can also lay foundations for developing locomotor neurotechnologies for rehabilitation or human performance. Technical barriers have largely prohibited neuroimaging during gait, but the portability and precise temporal resolution of non-invasive electroencephalography (EEG) have expanded human neuromotor research into increasingly dynamic tasks. In this narrative mini-review, we provide a (1) brief introduction and overview of modern neuroimaging technologies and then identify considerations for (2) mobile EEG hardware, (3) and data processing, (4) including technical challenges and possible solutions. Finally, we summarize (5) knowledge gained from human locomotor control studies that have used mobile EEG, and (6) discuss future directions for real-world neuroimaging research.

Flex-Printed Ear-EEG Sensors for Adequate Sleep Staging at Home

A comfortable, discrete and robust recording of the sleep EEG signal at home is a desirable goal but has been difficult to achieve. We investigate how well flex-printed electrodes are suitable for sleep monitoring tasks in a smartphone-based home environment. The cEEGrid ear-EEG sensor has already been tested in the laboratory for measuring night sleep. Here, 10 participants slept at home and were equipped with a cEEGrid and a portable amplifier (mBrainTrain, Serbia). In addition, the EEG of Fpz, EOG_L and EOG_R was recorded. All signals were recorded wirelessly with a smartphone. On average, each participant provided data for M = 7.48 h. An expert sleep scorer created hypnograms and annotated grapho-elements according to AASM based on the EEG of Fpz, EOG_L and EOG_R twice, which served as the baseline agreement for further comparisons. The expert scorer also created hypnograms using bipolar channels based on combinations of cEEGrid channels only, and bipolar cEEGrid channels complemented by EOG channels. A comparison of the hypnograms based on frontal electrodes with the ones based on cEEGrid electrodes (κ = 0.67) and the ones based on cEEGrid complemented by EOG channels (κ = 0.75) both showed a substantial agreement, with the combination including EOG channels showing a significantly better outcome than the one without (p = 0.006). Moreover, signal excerpts of the conventional channels containing grapho-elements were correlated with those of the cEEGrid in order to determine the cEEGrid channel combination that optimally represents the annotated grapho-elements. The results show that the grapho-elements were well-represented by the front-facing electrode combinations. The correlation analysis of the grapho-elements resulted in an average correlation coefficient of 0.65 for the most suitable electrode configuration of the cEEGrid. The results confirm that sleep stages can be identified with electrodes placement around the ear. This opens up opportunities for miniaturized ear-EEG systems that may be self-applied by users.

Performance of ECG-based seizure detection algorithms strongly depends on training and test conditions

Objective

To identify non‐EEG‐based signals and algorithms for detection of motor and non‐motor seizures in people lying in bed during video‐EEG (VEEG) monitoring and to test whether these algorithms work in freely moving people during mobile EEG recordings.

Methods

Data of three groups of adult people with epilepsy (PwE) were analyzed. Group 1 underwent VEEG with additional devices (accelerometry, ECG, electrodermal activity); group 2 underwent VEEG; and group 3 underwent mobile EEG recordings both including one‐lead ECG. All seizure types were analyzed. Feature extraction and machine‐learning techniques were applied to develop seizure detection algorithms. Performance was expressed as sensitivity, precision, F₁ score, and false positives per 24 hours.

Results

The algorithms were developed in group 1 (35 PwE, 33 seizures) and achieved best results (F₁ score 56%, sensitivity 67%, precision 45%, false positives 0.7/24 hours) when ECG features alone were used, with no improvement by including accelerometry and electrodermal activity. In group 2 (97 PwE, 255 seizures), this ECG‐based algorithm largely achieved the same performance (F₁ score 51%, sensitivity 39%, precision 73%, false positives 0.4/24 hours). In group 3 (30 PwE, 51 seizures), the same ECG‐based algorithm failed to meet up with the performance in groups 1 and 2 (F₁ score 27%, sensitivity 31%, precision 23%, false positives 1.2/24 hours). ECG‐based algorithms were also separately trained on data of groups 2 and 3 and tested on the data of the other groups, yielding maximal F₁ scores between 8% and 26%.

Significance

Our results suggest that algorithms based on ECG features alone can provide clinically meaningful performance for automatic detection of all seizure types. Our study also underscores that the circumstances under which such algorithms were developed, and the selection of the training and test data sets need to be considered and limit the application of such systems to unseen patient groups behaving in different conditions.

Feasibility of Repeated Assessment of Cognitive Function in Older Adults Using a Wireless, Mobile, Dry-EEG Headset and Tablet-Based Games

Access to affordable, objective and scalable biomarkers of brain function is needed to transform the healthcare burden of neuropsychiatric and neurodegenerative disease. Electroencephalography (EEG) recordings, both resting and in combination with targeted cognitive tasks, have demonstrated utility in tracking disease state and therapy response in a range of conditions from schizophrenia to Alzheimer's disease. But conventional methods of recording this data involve burdensome clinic visits, and behavioural tasks that are not effective in frequent repeated use. This paper aims to evaluate the technical and human-factors feasibility of gathering large-scale EEG using novel technology in the home environment with healthy adult users. In a large field study, 89 healthy adults aged 40-79 years volunteered to use the system at home for 12 weeks, 5 times/week, for 30 min/session. A 16-channel, dry-sensor, portable wireless headset recorded EEG while users played gamified cognitive and passive tasks through a tablet application, including tests of decision making, executive function and memory. Data was uploaded to cloud servers and remotely monitored via web-based dashboards. Seventy-eight participants completed the study, and high levels of adherence were maintained throughout across all age groups, with mean compliance over the 12-week period of 82% (4.1 sessions per week). Reported ease of use was also high with mean System Usability Scale scores of 78.7. Behavioural response measures (reaction time and accuracy) and EEG components elicited by gamified stimuli (P300, ERN, Pe and changes in power spectral density) were extracted from the data collected in home, across a wide range of ages, including older adult participants. Findings replicated well-known patterns of age-related change and demonstrated the feasibility of using low-burden, large-scale, longitudinal EEG measurement in community-based cohorts. This technology enables clinically relevant data to be recorded outside the lab/clinic, from which metrics underlying cognitive ageing could be extracted, opening the door to potential new ways of developing digital cognitive biomarkers for disorders affecting the brain.

Transcranial Auricular Vagus Nerve Stimulation (taVNS) and Ear-EEG: Potential for Closed-Loop Portable Non-invasive Brain Stimulation

No matter how hard we concentrate, our attention fluctuates – a fact that greatly affects our success in completing a current task. Here, we review work from two methods that, in a closed-loop manner, have the potential to ameliorate these fluctuations. Ear-EEG can measure electric brain activity from areas in or around the ear, using small and thus portable hardware. It has been shown to capture the state of attention with high temporal resolution. Transcutaneous auricular vagus nerve stimulation (taVNS) comes with the same advantages (small and light) and critically current research suggests that it is possible to influence ongoing brain activity that has been linked to attention. Following the review of current work on ear-EEG and taVNS we suggest that a combination of the two methods in a closed-loop system could serve as a potential application to modulate attention.

Feasibility of assessing brain activity using mobile, in-home collection of electroencephalography: methods and analysis

The last decade has seen increased availability of mobile electroencephalography (EEG). These mobile systems enable researchers to conduct data collection "in-context," reducing participant burden and potentially increasing diversity and representation of research samples. Our research team completed in-home data collection from more than 400 twelve-month-old infants from low-income backgrounds using a mobile EEG system. In this paper, we provide methodological and analytic guidance for collecting high-quality, mobile EEG in infants. Specifically, we offer insights and recommendations for equipment selection, data collection, and data analysis, highlighting important considerations for selecting a mobile EEG system. Examples include the size of the recording equipment, electrode type, reference types, and available montages. We also highlight important recommendations surrounding preparing a nonstandardized recording environment for EEG collection, obtaining informed consent from parents, instructions for parents during capping and recording, stimuli and task design, training researchers, and monitoring data as it comes in. Additionally, we provide access to the analysis code and demonstrate the robustness of the data from a recent study using this approach, in which 20 artifact-free epochs achieve good internal consistency reliability. Finally, we provide recommendations and publicly available resources for future studies aiming to collect mobile EEG.

Mobile brain/body imaging of landmark-based navigation with high-density EEG

Coupling behavioral measures and brain imaging in naturalistic, ecological conditions is key to comprehend the neural bases of spatial navigation. This highly integrative function encompasses sensorimotor, cognitive, and executive processes that jointly mediate active exploration and spatial learning. However, most neuroimaging approaches in humans are based on static, motion‐constrained paradigms and they do not account for all these processes, in particular multisensory integration. Following the Mobile Brain/Body Imaging approach, we aimed to explore the cortical correlates of landmark‐based navigation in actively behaving young adults, solving a Y‐maze task in immersive virtual reality. EEG analysis identified a set of brain areas matching state‐of‐the‐art brain imaging literature of landmark‐based navigation. Spatial behavior in mobile conditions additionally involved sensorimotor areas related to motor execution and proprioception usually overlooked in static fMRI paradigms. Expectedly, we located a cortical source in or near the posterior cingulate, in line with the engagement of the retrosplenial complex in spatial reorientation. Consistent with its role in visuo‐spatial processing and coding, we observed an alpha‐power desynchronization while participants gathered visual information. We also hypothesized behavior‐dependent modulations of the cortical signal during navigation. Despite finding few differences between the encoding and retrieval phases of the task, we identified transient time–frequency patterns attributed, for instance, to attentional demand, as reflected in the alpha/gamma range, or memory workload in the delta/theta range. We confirmed that combining mobile high‐density EEG and biometric measures can help unravel the brain structures and the neural modulations subtending ecological landmark‐based navigation.

An Analysis of the External Validity of EEG Spectral Power in an Uncontrolled Outdoor Environment during Default and Complex Neurocognitive States

Traditionally, quantitative electroencephalography (QEEG) studies collect data within controlled laboratory environments that limit the external validity of scientific conclusions. To probe these validity limits, we used a mobile EEG system to record electrophysiological signals from human participants while they were located within a controlled laboratory environment and an uncontrolled outdoor environment exhibiting several moderate background influences. Participants performed two tasks during these recordings, one engaging brain activity related to several complex cognitive functions (number sense, attention, memory, executive function) and the other engaging two default brain states. We computed EEG spectral power over three frequency bands (theta: 4-7 Hz, alpha: 8-13 Hz, low beta: 14-20 Hz) where EEG oscillatory activity is known to correlate with the neurocognitive states engaged by these tasks. Null hypothesis significance testing yielded significant EEG power effects typical of the neurocognitive states engaged by each task, but only a beta-band power difference between the two background recording environments during the default brain state. Bayesian analysis showed that the remaining environment null effects were unlikely to reflect measurement insensitivities. This overall pattern of results supports the external validity of laboratory EEG power findings for complex and default neurocognitive states engaged within moderately uncontrolled environments.

EEG in motion: Using an oddball task to explore motor interference in active skateboarding

Recent advancements in portable computer devices have opened new avenues in the study of human cognition outside research laboratories. This flexibility in methodology has led to the publication of several electroencephalography studies recording brain responses in real-world scenarios such as cycling and walking outside. In the present study, we tested the classic auditory oddball task while participants moved around an indoor running track using an electric skateboard. This novel approach allows for the study of attention in motion while virtually removing body movement. Using the skateboard auditory oddball paradigm, we found reliable and expected standard-target differences in the P3 and MMN/N2b event-related potentials. We also recorded baseline electroencephalography activity and found that, compared to this baseline, alpha power is attenuated in frontal and parietal regions during skateboarding. In order to explore the influence of motor interference in cognitive resources during skateboarding, we compared participants' preferred riding stance (baseline level of riding difficulty) versus their non-preferred stance (increased level of riding difficulty). We found that an increase in riding difficulty did not modulate the P3 and tonic alpha amplitude during skateboard motion. These results suggest that increases in motor demands might not lead to reductions in cognitive resources as shown in previous literature.

Using Muse: Rapid Mobile Assessment of Brain Performance

The advent of mobile electroencephalography (mEEG) has created a means for large scale collection of neural data thus affording a deeper insight into cognitive phenomena such as cognitive fatigue. Cognitive fatigue - a neural state that is associated with an increased incidence of errorful performance - is responsible for accidents on a daily basis which at times can cost human lives. To gain better insight into the neural signature of cognitive fatigue in the present study we used mEEG to examine the relationship between perceived cognitive fatigue and human-event related brain potentials (ERPs) and electroencephalographic (EEG) oscillations in a sample of 1,000 people. As a secondary goal, we wanted to further demonstrate the capability of mEEG to accurately measure ERP and EEG data. To accomplish these goals, participants performed a standard visual oddball task on an Apple iPad while EEG data were recorded from a Muse EEG headband. Counter to traditional EEG studies, experimental setup and data collection was completed in less than seven minutes on average. An analysis of our EEG data revealed robust N200 and P300 ERP components and neural oscillations in the delta, theta, alpha, and beta bands. In line with previous findings we observed correlations between ERP components and EEG power and perceived cognitive fatigue. Further, we demonstrate here that a linear combination of ERP and EEG features is a significantly better predictor of perceived cognitive fatigue than any ERP or EEG feature on its own. In sum, our results provide validation of mEEG as a viable tool for research and provide further insight into the impact of cognitive fatigue on the human brain.

Overground Walking Decreases Alpha Activity and Entrains Eye Movements in Humans

Experiments in animal models have shown that running increases neuronal activity in early visual areas in light as well as in darkness. This suggests that visual processing is influenced by locomotion independent of visual input. Combining mobile electroencephalography, motion- and eye-tracking, we investigated the influence of overground free walking on cortical alpha activity (~10 Hz) and eye movements in healthy humans. Alpha activity has been considered a valuable marker of inhibition of sensory processing and shown to negatively correlate with neuronal firing rates. We found that walking led to a decrease in alpha activity over occipital cortex compared to standing. This decrease was present during walking in darkness as well as during light. Importantly, eye movements could not explain the change in alpha activity. Nevertheless, we found that walking and eye related movements were linked. While the blink rate increased with increasing walking speed independent of light or darkness, saccade rate was only significantly linked to walking speed in the light. Pupil size, on the other hand, was larger during darkness than during light, but only showed a modulation by walking in darkness. Analyzing the effect of walking with respect to the stride cycle, we further found that blinks and saccades preferentially occurred during the double support phase of walking. Alpha power, as shown previously, was lower during the swing phase than during the double support phase. We however could exclude the possibility that the alpha modulation was introduced by a walking movement induced change in electrode impedance. Overall, our work indicates that the human visual system is influenced by the current locomotion state of the body. This influence affects eye movement pattern as well as neuronal activity in sensory areas and might form part of an implicit strategy to optimally extract sensory information during locomotion.

Eye movement-related brain potentials during assisted navigation in real-world environments

Conducting neuroscience research in the real-world remains challenging because of movement- and environment-related artifacts as well as missing control over stimulus presentation. The present study overcame these restrictions by mobile electroencephalography (EEG) and data-driven analysis approaches during a real-world navigation task. During assisted navigation through an unfamiliar city environment, participants received either standard or landmark-based auditory navigation instructions. EEG data were recorded continuously during navigation. Saccade- and blink-events as well as gait-related EEG activity were extracted from sensor level data. Brain activity associated with the navigation task was identified by subsequent source-based cleaning of non-brain activity and unfolding of overlapping event-related potentials. When navigators received landmark-based instructions compared to those receiving standard navigation instructions, the blink-related brain potentials during navigation revealed higher amplitudes at fronto-central leads in a time window starting at 300 ms after blinks, which was accompanied by improved spatial knowledge acquisition tested in follow-up spatial tasks. Replicating improved spatial knowledge acquisition from previous experiments, the present study revealed eye movement-related brain potentials to point to the involvement of higher cognitive processes and increased processing of incoming information during periods of landmark-based instructions. The study revealed neuronal correlates underlying visuospatial information processing during assisted navigation in the real-world providing a new analysis approach for neuroscientific research in freely moving participants in uncontrollable real-world environments.

Does the electrode amplification style matter? A comparison of active and passive EEG system configurations during standing and walking

It has been stated that active-transmission electrodes should improve signal quality in mobile EEG recordings. However, few studies have directly compared active- and passive-transmission electrodes during a mobile task. In this repeated measurement study, we investigated the performance of active and passive signal transmission electrodes with the same amplifier system in their respective typical configurations, during a mobile auditory task. The task was an auditory discrimination (1,000 vs. 800 Hz; counterbalanced) oddball task using approximately 560 trials (15% targets) for each condition. Eighteen participants performed the auditory oddball task both while standing and walking in an outdoor environment. While walking, there was a significant decrease in P3 amplitude, post-trial rejection trial numbers, and signal-to-noise ratio (SNR). No significant differences were found in signal quality between the two electrode configurations. SNR and P3 amplitude were test-retest reliable between recordings. We conclude that adequate use of a passive EEG electrode system achieves signal quality equivalent to that of an active system during a mobile task.

Wireless Programmable Recording and Stimulation of Deep Brain Activity in Freely Moving Humans

Uncovering the neural mechanisms underlying human natural ambulatory behavior is a major challenge for neuroscience. Current commercially available implantable devices that allow for recording and stimulation of deep brain activity in humans can provide invaluable intrinsic brain signals but are not inherently designed for research and thus lack flexible control and integration with wearable sensors. We developed a mobile deep brain recording and stimulation (Mo-DBRS) platform that enables wireless and programmable intracranial electroencephalographic recording and electrical stimulation integrated and synchronized with virtual reality/augmented reality (VR/AR) and wearables capable of external measurements (e.g., motion capture, heart rate, skin conductance, respiration, eye tracking, and scalp EEG). When used in freely moving humans with implanted neural devices, this platform is adaptable to ecologically valid environments conducive to elucidating the neural mechanisms underlying naturalistic behaviors and to the development of viable therapies for neurologic and psychiatric disorders.

Aiming at ecological validity-Midfrontal theta oscillations in a toy gun shooting task

Laboratory electroencephalography (EEG) studies have already provided important insights into the neuronal mechanisms of performance monitoring. However, to our knowledge no study so far has examined neuronal correlates of performance monitoring using an ecologically valid task outside a typical laboratory setting. Therefore, we examined midfrontal theta and the feedback-related negativity (FRN) using mobile EEG in a physical shooting task within an ecologically valid environment with highly dynamical visual feedback. Participants shot a target using a toy gun while moving and looking around freely. Shots that missed the target evoked stronger midfrontal theta activity than hits and this response was rather phase-unlocked. There was no difference between misses and hits in the FRN. The results raise the question whether the absence of certain ERP components like the FRN could be due to methodological reasons or to the fact that partially different neuronal processes may be activated in the laboratory as compared to more ecologically valid tasks. Overall, our results indicate that crucial neurocognitive processes of performance monitoring can be assessed in highly dynamic and ecologically valid settings by mobile EEG.

A walk in the park? Characterizing gait-related artifacts in mobile EEG recordings

Brain activity during natural walking outdoors can be captured using mobile electroencephalography (EEG). However, EEG recorded during gait is confounded with artifacts from various sources, possibly obstructing the interpretation of brain activity patterns. Currently, there is no consensus on how the amount of artifact present in these recordings should be quantified, or is there a systematic description of gait artifact properties. In the current study, we expand several features into a seven-dimensional footprint of gait-related artifacts, combining features of time, time-frequency, spatial, and source domains. EEG of N = 26 participants was recorded while standing and walking outdoors. Footprints of gait-related artifacts before and after two different artifact attenuation strategies (after artifact subspace reconstruction (ASR) and after subsequent independent component analysis [ICA]) were systematically different. We also evaluated topographies, morphologies, and signal-to-noise ratios (SNR) of button-press event-related potentials (ERP) before and after artifact handling, to confirm gait-artifact reduction specificity. Morphologies and SNR remained unchanged after artifact attenuation, whereas topographies improved in quality. Our results show that the footprint can provide a detailed assessment of gait-related artifacts and can be used to estimate the sensitivity of different artifact reduction strategies. Moreover, the analysis of button-press ERPs demonstrated its specificity, as processing did not only reduce gait-related artifacts but ERPs of interest remained largely unchanged. We conclude that the proposed footprint is well suited to characterize individual differences in gait-related artifact extent. In the future, it could be used to compare and optimize recording setups and processing pipelines comprehensively.

Analyzing and Decoding Natural Reach-and-Grasp Actions Using Gel, Water and Dry EEG Systems

Reaching and grasping is an essential part of everybody's life, it allows meaningful interaction with the environment and is key to independent lifestyle. Recent electroencephalogram (EEG)-based studies have already shown that neural correlates of natural reach-and-grasp actions can be identified in the EEG. However, it is still in question whether these results obtained in a laboratory environment can make the transition to mobile applicable EEG systems for home use. In the current study, we investigated whether EEG-based correlates of natural reach-and-grasp actions can be successfully identified and decoded using mobile EEG systems, namely the water-based EEG-Versatile ^TM system and the dry-electrodes EEG-Hero ^TM headset. In addition, we also analyzed gel-based recordings obtained in a laboratory environment (g.USBamp/g.Ladybird, gold standard), which followed the same experimental parameters. For each recording system, 15 study participants performed 80 self-initiated reach-and-grasp actions toward a glass (palmar grasp) and a spoon (lateral grasp). Our results confirmed that EEG-based correlates of reach-and-grasp actions can be successfully identified using these mobile systems. In a single-trial multiclass-based decoding approach, which incorporated both movement conditions and rest, we could show that the low frequency time domain (LFTD) correlates were also decodable. Grand average peak accuracy calculated on unseen test data yielded for the water-based electrode system 62.3% (9.2% STD), whereas for the dry-electrodes headset reached 56.4% (8% STD). For the gel-based electrode system 61.3% (8.6% STD) could be achieved. To foster and promote further investigations in the field of EEG-based movement decoding, as well as to allow the interested community to make their own conclusions, we provide all datasets publicly available in the BNCI Horizon 2020 database (http://bnci-horizon-2020.eu/database/data-sets).

A Protocol for Comparing Dry and Wet EEG Electrodes During Sleep

Background

Sleep is commonly assessed by recording the electroencephalogram (EEG) of the sleeping brain. As sleep assessments in a lab environment are cumbersome for both the participant and researcher, it would be highly desirable to record sleep EEG with a user-friendly and mobile device. Dry electrodes that are reusable, low-cost, and easy to apply would be an essential component of such a device. In this study, we developed a testing protocol to investigate the performance of novel flat-type dry electrodes for sleep EEG recordings in free-living conditions.

Methods

Overnight sleep EEG, electrooculogram and electromyogram of four young and healthy participants were recorded at home. Two identical ambulatory recording devices, one using novel flat-type dry electrodes, the other using self-adhesive pre-gelled electrodes, simultaneously recorded sleep EEG. Between both electrode types, we then compared the signal quality, the incidence of artifacts, the sensitivity, specificity and inter-scoring reliability (Cohen’s kappa) of sleep staging, as well as the agreement of important characteristics of sleep-specific EEG microstructure features, such as slow waves (0.5–4 Hz) and sleep spindles (10–16 Hz).

Results

Our testing protocol comprehensively compared the two electrode types on a macro- and microstructure level of sleep. The dry and pre-gelled electrodes both had comparable signal quality and sleep staging was feasible with both electrodes. Also, slow-wave and spindle characteristics were similar. However, sweat artifacts were more prevalent in the flat-type dry electrodes.

Conclusion

With a reliable testing protocol, the performance of dry electrodes can be compared to reference technologies and objectively assessed also in free-living conditions.

Mobile brain/body Imaging in dance: A dynamic transdisciplinary field for applied research

Neuroscience of dance is an emerging field with important applications related to health and well-being, as dance has shown potential to foster adaptive neuroplasticity and is increasingly popular as a therapeutic activity or adjunct therapy for people living with conditions such as Parkinson's and Alzheimer's diseases. However, the multimodal nature of dance presents challenges to researchers aiming to identify mechanisms involved when dance is used to combat neurodegeneration or support healthy ageing. Requiring simultaneous engagement of motor and cognitive domains, dancing includes coordination of systems involved in timing, memory and spatial learning. Studies on dance to this point rely primarily on assessments of brain dynamics and structure through pre/post-tests or studies on expertise, as traditional brain imaging modalities restrict participant movement to avoid movement-related artefacts. In this paper, we describe the process of designing and implementing a study that uses mobile brain/body imaging (MoBI) to investigate real-time changes in brain dynamics and behaviour during the process of learning and performing a novel dance choreography. We show the potential for new insights to emerge from the coordinated collection of movement and brain-based data, and the implications of these in an emerging field whose medium is motion.

Challenge Accepted? Individual Performance Gains for Motor Imagery Practice with Humanoid Robotic EEG Neurofeedback

Optimizing neurofeedback (NF) and brain–computer interface (BCI) implementations constitutes a challenge across many fields and has so far been addressed by, among others, advancing signal processing methods or predicting the user’s control ability from neurophysiological or psychological measures. In comparison, how context factors influence NF/BCI performance is largely unexplored. We here investigate whether a competitive multi-user condition leads to better NF/BCI performance than a single-user condition. We implemented a foot motor imagery (MI) NF with mobile electroencephalography (EEG). Twenty-five healthy, young participants steered a humanoid robot in a single-user condition and in a competitive multi-user race condition using a second humanoid robot and a pseudo competitor. NF was based on 8–30 Hz relative event-related desynchronization (ERD) over sensorimotor areas. There was no significant difference between the ERD during the competitive multi-user condition and the single-user condition but considerable inter-individual differences regarding which condition yielded a stronger ERD. Notably, the stronger condition could be predicted from the participants’ MI-induced ERD obtained before the NF blocks. Our findings may contribute to enhance the performance of NF/BCI implementations and highlight the necessity of individualizing context factors.

In-ear-EEG - a portable platform for home monitoring

There are many useful medical treatment devices today, which are indispensable in health care. However, in some emergency situations and in prehospital care mobile, easy-to-use devices could further improve the patient-centred care. For example, a mobile, easy-to-use home-monitoring EEG-system would be useful for monitoring diseases like epilepsy and for treating diseases like attention deficit disorder (ADHD) using biofeedback. Such a device should be equipped with the ability to start self-performed by user recordings and provide high signal quality, while having an affordable price. Here, we used in-ear-EEG technology and state of the art electronic components to develop such a system. This paper presents a portable, all-in-one EEG-system, capable to record biosignals on the external ear. An amplifier was developed with ADS1299 and optimised to be coupled with a smartphone. The system has a low price and at the same time provides high signal quality, has very effective common-mode-rejection, performs a fast cold start and shows low power consumptions which ensures a long time of operation. The system is easy to use and could be self-mounted and controlled by unskilled users as well. Results of test measurements are compared to a conventional EEG-System and show comparable records results quality.

Sensorimotor brain dynamics reflect architectural affordances

Anticipating meaningful actions in the environment is an essential function of the brain. Such predictive mechanisms originate from the motor system and allow for inferring actions from environmental affordances, and the potential to act within a specific environment. Using architecture, we provide a unique perspective on the ongoing debate in cognitive neuroscience and philosophy on whether cognition depends on movement or is decoupled from our physical structure. To investigate cognitive processes associated with architectural affordances, we used a mobile brain/body imaging approach recording brain activity synchronized to head-mounted displays. Participants perceived and acted on virtual transitions ranging from nonpassable to easily passable. We found that early sensory brain activity, on revealing the environment and before actual movement, differed as a function of affordances. In addition, movement through transitions was preceded by a motor-related negative component that also depended on affordances. Our results suggest that potential actions afforded by an environment influence perception.

A Riemannian Modification of Artifact Subspace Reconstruction for EEG Artifact Handling

Artifact Subspace Reconstruction (ASR) is an adaptive method for the online or offline correction of artifacts comprising multichannel electroencephalography (EEG) recordings. It repeatedly computes a principal component analysis (PCA) on covariance matrices to detect artifacts based on their statistical properties in the component subspace. We adapted the existing ASR implementation by using Riemannian geometry for covariance matrix processing. EEG data that were recorded on smartphone in both outdoors and indoors conditions were used for evaluation (N = 27). A direct comparison between the original ASR and Riemannian ASR (rASR) was conducted for three performance measures: reduction of eye-blinks (sensitivity), improvement of visual-evoked potentials (VEPs) (specificity), and computation time (efficiency). Compared to ASR, our rASR algorithm performed favorably on all three measures. We conclude that rASR is suitable for the offline and online correction of multichannel EEG data acquired in laboratory and in field conditions.

Machine-learning-derived sleep-wake staging from around-the-ear electroencephalogram outperforms manual scoring and actigraphy

Quantification of sleep is important for the diagnosis of sleep disorders and sleep research. However, the only widely accepted method to obtain sleep staging is by visual analysis of polysomnography (PSG), which is expensive and time consuming. Here, we investigate automated sleep scoring based on a low-cost, mobile electroencephalogram (EEG) platform consisting of a lightweight EEG amplifier combined with flex-printed cEEGrid electrodes placed around the ear, which can be implemented as a fully self-applicable sleep system. However, cEEGrid signals have different amplitude characteristics to normal scalp PSG signals, which might be challenging for visual scoring. Therefore, this study evaluates the potential of automatic scoring of cEEGrid signals using a machine learning classifier ("random forests") and compares its performance with manual scoring of standard PSG. In addition, the automatic scoring of cEEGrid signals is compared with manual annotation of the cEEGrid recording and with simultaneous actigraphy. Acceptable recordings were obtained in 15 healthy volunteers (aged 35 ± 14.3 years) during an extended nocturnal sleep opportunity, which induced disrupted sleep with a large inter-individual variation in sleep parameters. The results demonstrate that machine-learning-based scoring of around-the-ear EEG outperforms actigraphy with respect to sleep onset and total sleep time assessments. The automated scoring outperforms human scoring of cEEGrid by standard criteria. The accuracy of machine-learning-based automated scoring of cEEGrid sleep recordings compared with manual scoring of standard PSG was satisfactory. The findings show that cEEGrid recordings combined with machine-learning-based scoring holds promise for large-scale sleep studies.

Out and about: Subsequent memory effect captured in a natural outdoor environment with smartphone EEG

Spatiotemporal context plays an important role in episodic memory. While temporal context effects have been frequently studied in the laboratory, ecologically valid spatial context manipulations are difficult to implement in stationary conditions. We investigated whether the neural correlates of successful encoding (subsequent memory effect) can be captured in a real-world environment. An off-the-shelf Android smartphone was used for wireless mobile EEG acquisition and stimulus presentation. Participants encoded single words, each of which was presented at a different location on a university campus. Locations were approximately 10-12 m away from each other, half of them with striking features (landmarks) nearby. We predicted landmarks would improve recall performance. After a first free recall task of verbal stimuli indoors, participants performed a subsequent recall outdoors, in which words and locations were recalled. As predicted, significantly more words presented at landmark locations as well as significantly more landmark than nonlandmark locations were recalled. ERP analysis yielded a larger posterior positive deflection during encoding for hits compared to misses in the 400-800 ms interval. Likewise, time-frequency analysis revealed a significant difference during encoding for hits compared to misses in the form of stronger alpha (200-300 ms) and theta (300-400 ms) power increases. Our results confirm that a vibrant spatial context is beneficial in episodic memory processing and that the underlying neural correlates can be captured with unobtrusive smartphone EEG technology. The advent of mobile EEG technology promises to unveil the relevance of natural physical activity and natural environments on memory.

Tracking Economic Value of Products in Natural Settings: A Wireless EEG Study

Economic decision making refers to the process of individuals translating their preference into subjective value (SV). Little is known about the dynamics of the neural processes that underpin this form of value-based decision making and no studies have investigated these processes outside of controlled laboratory settings. The current study investigated the spatio-temporal dynamics that accompany economic valuation of products using mobile electroencephalography (EEG) and eye tracking techniques. Participants viewed and rated images of household products in a gallery setting while EEG and eye tracking data were collected wirelessly. A Becker-DeGroot-Marschak (BDM) auction task was subsequently used to quantify the individual's willingness to pay (WTP) for each product. WTP was used to classify products into low, low medium, high medium and high economic value conditions. Eye movement related potentials (EMRP) were examined, and independent component analysis (ICA) was used to separate sources of activity from grand averaged EEG data. Four independent components (ICs) of EMRPs were modulated by WTP (i.e., SV) in the latency range of 150-250 ms. Of the four value-sensitive ICs, one IC displayed enhanced amplitude for all value conditions excluding low value, and another IC presented enhanced amplitude for low value products only. The remaining two value-sensitive ICs resolved inter-mediate levels of SV. Our study quantified, for the first time, the neural processes involved in economic value based decisions in a natural setting. Results suggest that multiple spatio-temporal brain activation patterns mediate the attention and aversion of products which could reflect an early valuation system. The EMRP parietal P200 component could reflect an attention allocation mechanism that separates the lowest-value products (IC7) from products of all other value (IC4), suggesting that low-value items are categorized early on as being aversive. While none of the ICs showed linear amplitude changes that parallel SV's of products, results suggest that a combination of multiple components may sub-serve a fine-grained resolution of the SV of products.

Threat-conditioned contexts modulate the late positive potential to faces-A mobile EEG/virtual reality study

In everyday life, the motivational value of faces is bound to the contexts in which faces are perceived. Electrophysiological studies have demonstrated that inherent negatively valent contexts modulate cortical face processing as assessed with ERP components. However, it is not well understood whether learned (rather than inherent) and three-dimensional aversive contexts similarly modulate the neural processing of faces. Using full immersive virtual reality (VR) and mobile EEG techniques, 25 participants underwent a differential fear conditioning paradigm, in which one virtual room was paired with an aversive noise burst (threat context) and another with a nonaversive noise burst (safe context). Subsequently, avatars with neutral or angry facial expressions were presented in the threat and safe contexts while EEG was recorded. Analysis of the late positive potential (LPP), which presumably indicates motivational salience, revealed a significant interaction of context (threat vs. safe) and face type (neutral vs. angry). Neutral faces evoked increased LPP amplitudes in threat versus safe contexts, while angry faces evoked increased early LPP amplitudes regardless of context. In addition to indicating that threat-conditioned contexts alter the processing of ambiguous faces, the present study demonstrates the successful integration of EEG and VR with particular relevance for affective neuroscience research.

Signal Quality Evaluation of Emerging EEG Devices

Electroencephalogram (EEG) registration as a direct measure of brain activity has unique potentials. It is one of the most reliable and predicative indicators when studying human cognition, evaluating a subject's health condition, or monitoring their mental state. Unfortunately, standard signal acquisition procedures limit the usability of EEG devices and narrow their application outside the lab. Emerging sensor technology allows gel-free EEG registration and wireless signal transmission. Thus, it enables quick and easy application of EEG devices by users themselves. Although a main requirement for the interpretation of an EEG is good signal quality, there is a lack of research on this topic in relation to new devices. In our work, we compared the signal quality of six very different EEG devices. On six consecutive days, 24 subjects wore each device for 60 min and completed tasks and games on the computer. The registered signals were evaluated in the time and frequency domains. In the time domain, we examined the percentage of artifact-contaminated EEG segments and the signal-to-noise ratios. In the frequency domain, we focused on the band power variation in relation to task demands. The results indicated that the signal quality of a mobile, gel-based EEG system could not be surpassed by that of a gel-free system. However, some of the mobile dry-electrode devices offered signals that were almost comparable and were very promising. This study provided a differentiated view of the signal quality of emerging mobile and gel-free EEG recording technology and allowed an assessment of the functionality of the new devices. Hence, it provided a crucial prerequisite for their general application, while simultaneously supporting their further development.

High-Intensity Chronic Stroke Motor Imagery Neurofeedback Training at Home: Three Case Reports

Motor imagery (MI) with neurofeedback has been suggested as promising for motor recovery after stroke. Evidence suggests that regular training facilitates compensatory plasticity, but frequent training is difficult to integrate into everyday life. Using a wireless electroencephalogram (EEG) system, we implemented a frequent and efficient neurofeedback training at the patients' home. Aiming to overcome maladaptive changes in cortical lateralization patterns we presented a visual feedback, representing the degree of contralateral sensorimotor cortical activity and the degree of sensorimotor cortex lateralization. Three stroke patients practiced every other day, over a period of 4 weeks. Training-related changes were evaluated on behavioral, functional, and structural levels. All 3 patients indicated that they enjoyed the training and were highly motivated throughout the entire training regime. EEG activity induced by MI of the affected hand became more lateralized over the course of training in all three patients. The patient with a significant functional change also showed increased white matter integrity as revealed by diffusion tensor imaging, and a substantial clinical improvement of upper limb motor functions. Our study provides evidence that regular, home-based practice of MI neurofeedback has the potential to facilitate cortical reorganization and may also increase associated improvements of upper limb motor function in chronic stroke patients.

Concealed Around-the-Ear EEG Captures Cognitive Processing in a Visual Simon Task

In theory, miniaturized systems such as the around-the-ear electrode arrays (cEEGrids) enable mobile monitoring of the electroencephalogram (EEG) in a variety of real life situations without interfering with the natural setting. However, the research benefit of such cEEGrid recordings critically depends on their validity. To investigate whether visual and motor processing are reflected in the cEEGrid-EEG, a direct comparison of EEG that was concurrently recorded with the cEEGrids and with a high-density cap setup was conducted. Thirteen participants performed a classic Simon task in which letters were presented laterally and a lateralized choice response was executed. N1, P1 and P300 event-related potential (ERP) waveforms were extracted from cEEGrid-EEG: they were found to be strongly correlated with corresponding waveforms extracted from cap-EEG but with lower signal strength and lower signal-to-noise-ratio (SNR). Event-related lateralizations (ERLs) recorded at posterior scalp sites were well reflected in middle cEEGrid pairs. Moreover, the effect size of the Simon correspondence effect on the extracted ERLs was similar between the two systems. However, lateralizations at central cap sites were less well reflected in the cEEGrid-EEG indicating a difficulty in capturing motor response preparation and execution. These results show that well-described visual and cognitive ERPs and ERLs can be measured using the cEEGrids, while motor-related cortical potentials are not well captured. This study further demonstrates the potential and possible limitations of unobtrusive cEEGrid-EEG recordings.

Concealed, Unobtrusive Ear-Centered EEG Acquisition: cEEGrids for Transparent EEG

Electroencephalography (EEG) is an important clinical tool and frequently used to study the brain-behavior relationship in humans noninvasively. Traditionally, EEG signals are recorded by positioning electrodes on the scalp and keeping them in place with glue, rubber bands, or elastic caps. This setup provides good coverage of the head, but is impractical for EEG acquisition in natural daily-life situations. Here, we propose the transparent EEG concept. Transparent EEG aims for motion tolerant, highly portable, unobtrusive, and near invisible data acquisition with minimum disturbance of a user's daily activities. In recent years several ear-centered EEG solutions that are compatible with the transparent EEG concept have been presented. We discuss work showing that miniature electrodes placed in and around the human ear are a feasible solution, as they are sensitive enough to pick up electrical signals stemming from various brain and non-brain sources. We also describe the cEEGrid flex-printed sensor array, which enables unobtrusive multi-channel EEG acquisition from around the ear. In a number of validation studies we found that the cEEGrid enables the recording of meaningful continuous EEG, event-related potentials and neural oscillations. Here, we explain the rationale underlying the cEEGrid ear-EEG solution, present possible use cases and identify open issues that need to be solved on the way toward transparent EEG.

Cortical Sensitivity to Guitar Note Patterns: EEG Entrainment to Repetition and Key

Music is ubiquitous throughout recent human culture, and many individual's have an innate ability to appreciate and understand music. Our appreciation of music likely emerges from the brain's ability to process a series of repeated complex acoustic patterns. In order to understand these processes further, cortical responses were measured to a series of guitar notes presented with a musical pattern or without a pattern. ERP responses to individual notes were measured using a 24 electrode Bluetooth mobile EEG system (Smarting mBrainTrain) while 13 healthy non-musicians listened to structured (i.e., within musical keys and with repetition) or random sequences of guitar notes for 10 min each. We demonstrate an increased amplitude to the ERP that appears ~200 ms to notes presented within the musical sequence. This amplitude difference between random notes and patterned notes likely reflects individual's cortical sensitivity to guitar note patterns. These amplitudes were compared to ERP responses to a rare note embedded within a stream of frequent notes to determine whether the sensitivity to complex musical structure overlaps with the sensitivity to simple irregularities reflected in traditional auditory oddball experiments. Response amplitudes to the negative peak at ~175 ms are statistically correlated with the mismatch negativity (MMN) response measured to a rare note presented among a series of frequent notes (i.e., in a traditional oddball sequence), but responses to the subsequent positive peak at ~200 do not show a statistical relationship with the P300 response. Thus, the sensitivity to musical structure identified to 4 Hz note patterns appears somewhat distinct from the sensitivity to statistical regularities reflected in the traditional "auditory oddball" sequence. Overall, we suggest that this is a promising approach to examine individual's sensitivity to complex acoustic patterns, which may overlap with higher level cognitive processes, including language.

The N400 Effect during Speaker-Switch-Towards a Conversational Approach of Measuring Neural Correlates of Language

Language occurs naturally in conversations. However, the study of the neural underpinnings of language has mainly taken place in single individuals using controlled language material. The interactive elements of a conversation (e.g., turn-taking) are often not part of neurolinguistic setups. The prime reason is the difficulty to combine open unrestricted conversations with the requirements of neuroimaging. It is necessary to find a trade-off between the naturalness of a conversation and the restrictions imposed by neuroscientific methods to allow for ecologically more valid studies. Here, we make an attempt to study the effects of a conversational element, namely turn-taking, on linguistic neural correlates, specifically the N400 effect. We focus on the physiological aspect of turn-taking, the speaker-switch, and its effect on the detectability of the N400 effect. The N400 event-related potential reflects expectation violations in a semantic context; the N400 effect describes the difference of the N400 amplitude between semantically expected and unexpected items. Sentences with semantically congruent and incongruent final words were presented in two turn-taking modes: (1) reading aloud first part of the sentence and listening to speaker-switch for the final word, and (2) listening to first part of the sentence and speaker-switch for the final word. A significant N400 effect was found for both turn-taking modes, which was not influenced by the mode itself. However, the mode significantly affected the P200, which was increased for the reading aloud mode compared to the listening mode. Our results show that an N400 effect can be detected during a speaker-switch. Speech articulation (reading aloud) before the analyzed sentence fragment did also not impede the N400 effect detection for the final word. The speaker-switch, however, seems to influence earlier components of the electroencephalogram, related to processing of salient stimuli. We conclude that the N400 can effectively be used to study neural correlates of language in conversational approaches including speaker-switches.