Deep Convolutional Neural Networks for Feature-Less Automatic Classification of Independent Components in Multi-Channel Electrophysiological Brain Recordings

Shared spectral fingerprints of temporal memory precision and representation of the temporal structure of complex narratives

The ability to date events is fundamental to episodic memory. Separate lines of fMRI research have explored the neurobiological mechanisms underlying temporal precision and the representation of temporal structure in complex events. The present EEG study examined the oscillatory dynamics of both processes in participants performing a timeline positioning task with movie scenes. Multivariate analyses identified a high-beta/low-gamma electrophysiological signature of temporal precision during timeline presentation, involving a right-lateralized network. An independent representation similarity analysis revealed a strong coupling between behavioral and neural distance between pairs of movie parts in the same time-frequency band as the precision effect. Crucially, participants with higher temporal precision showed a stronger correlation between behavioral and neural distance, reinforcing the link between brain signals related to precision and temporal structure representation. These findings support the idea of a systematic temporal organization of experiences, which plays a role in guiding inferential processes.

Multi-modality NDE fusion using encoder-decoder networks for identify multiple neurological disorders from EEG signals

BackgroundThe complexity and diversity of brain activity patterns make it difficult to accurately diagnose neurological disorders such epilepsy, Parkinson's disease, schizophrenia, stroke, and Alzheimer's disease. Integrated and effective analysis of multiple data sources is often beyond the scope of traditional diagnostic procedures. With the use of multi-modal data, recent developments in neural network approaches present encouraging opportunities for raising diagnostic accuracy.ObjectivesA novel approach has been proposed toward the integration of different Nondestructive Evaluation data with EEG signals for improving the diagnosis of neurological disorders such as stroke, epilepsy, Parkinson's disease, and schizophrenia, by leveraging advanced neural network techniques in order to improve the identification and correlation of shared latent features across heterogeneous NDE datasets.MethodsWe determined the 2D scalogram images using a specific encoder-decoder neural network after transforming the EEG signals using wavelet signal processing. Several NDE data types can be easily integrated for thorough analysis due to this network's ability to extract and correlate important aspects from each form of data. Aiming to uncover common patterns indicating of neurological disorders, the technique was evaluated on datasets containing EEG signals and corresponding NDE data.ResultsOur method demonstrated a significant improvement in diagnostic accuracy and efficiency. The encoder-decoder network effectively identified shared latent features across the heterogeneous NDE datasets, leading to more precise and reliable diagnoses. The fusion of multi-modality NDE data with EEG signals provided a robust framework for the automatic identification of multiple neurological disorders.ConclusionsThis innovative approach represents a substantial advancement in the field of neurological disorder diagnosis. By integrating diverse NDE data with EEG signals through advanced neural network techniques, we have developed a method that enhances the accuracy and efficiency of diagnosing multiple neurological conditions. This fusion of multi-modality data has the potential to revolutionize current diagnostic practices in neurology, paving the way for more precise and automated identification of neurological disorders.

Brain dynamics of visual anticipation during spatial occlusion tasks in expert tennis players

Stimulus identification and action outcome understanding for a rapid and accurate response selection, play a fundamental role in racquet sports. Here, we investigated the neurodynamics of visual anticipation in tennis manipulating the postural and kinematic information associated with the body of opponents by means of a spatial occlusion protocol. Event Related Potentials (ERPs) were evaluated in two groups of professional tennis players (N = 37) with different levels of expertise, while they observed pictures of opponents and predicted the landing position as fast and accurately as possible. The observed action was manipulated by deleting different body districts of the opponent (legs, ball, racket and arm, trunk). Full body image (no occlusion) was used as control condition. The worst accuracy and the slowest response time were observed in the occlusion of trunk and ball. The former was associated with a reduced amplitude of the ERP components likely linked to body processing (the N1 in the right hemisphere) and visual-motor integration awareness (the pP1), as well as with an increase of the late frontal negativity (the pN2), possibly reflecting an effort by the insula to recover and/or complete the most correct sensory-motor representation. In both occlusions, a decrease in the pP2 may reflect an impairment of decisional processes upon action execution following sensory evidence accumulation. Enhanced amplitude of the P3 and the pN2 components were found in more experienced players, suggesting a greater allocation of resources in the process connecting sensory encoding and response execution, and sensory-motor representation.

Yield of EEG features as markers of disease severity in amyotrophic lateral sclerosis: a pilot study

OBJECTIVE

To clarify the role of electroencephalography (EEG) as a promising marker of severity in amyotrophic lateral sclerosis (ALS). We characterized the brain spatio-temporal patterns activity at rest by means of both spectral band powers and EEG microstates and correlated these features with clinical scores.

METHODS

Eyes closed EEG was acquired in 15 patients with ALS and spectral band power was calculated in frequency bands, defined on the basis of individual alpha frequency (IAF): delta-theta band (1-7 Hz); low alpha (IAF - 2 Hz - IAF); high alpha (IAF - IAF + 2 Hz); beta (13 - 25 Hz). EEG microstate metrics (duration, occurrence, and coverage) were also evaluated. Spectral band powers and microstate metrics were correlated with several clinical scores of disabilities and disease progression. As a control group, 15 healthy volunteers were enrolled.

RESULTS

The beta-band power in motor/frontal regions was higher in patients with higher disease burden, negatively correlated with clinical severity scores and positively correlated with disease progression. Overall microstate duration was longer and microstate occurrence was lower in patients than in controls. Longer duration was correlated with a worse clinical status.

CONCLUSIONS

Our results showed that beta-band power and microstate metrics may be good candidates of disease severity in ALS. Increased beta and longer microstate duration in clinically worse patients suggest a possible impairment of both motor and non-motor network activities to fast modify their status. This can be interpreted as an attempt in ALS patients to compensate the disability but resulting in an ineffective and probably maladaptive behavior.

ABOT: an open-source online benchmarking tool for machine learning-based artefact detection and removal methods from neuronal signals

Brain signals are recorded using different techniques to aid an accurate understanding of brain function and to treat its disorders. Untargeted internal and external sources contaminate the acquired signals during the recording process. Often termed as artefacts, these contaminations cause serious hindrances in decoding the recorded signals; hence, they must be removed to facilitate unbiased decision-making for a given investigation. Due to the complex and elusive manifestation of artefacts in neuronal signals, computational techniques serve as powerful tools for their detection and removal. Machine learning (ML) based methods have been successfully applied in this task. Due to ML's popularity, many articles are published every year, making it challenging to find, compare and select the most appropriate method for a given experiment. To this end, this paper presents ABOT (Artefact removal Benchmarking Online Tool) as an online benchmarking tool which allows users to compare existing ML-driven artefact detection and removal methods from the literature. The characteristics and related information about the existing methods have been compiled as a knowledgebase (KB) and presented through a user-friendly interface with interactive plots and tables for users to search it using several criteria. Key characteristics extracted from over 120 articles from the literature have been used in the KB to help compare the specific ML models. To comply with the FAIR (Findable, Accessible, Interoperable and Reusable) principle, the source code and documentation of the toolbox have been made available via an open-access repository.

Deep Learning Adapted to Differential Neural Networks Used as Pattern Classification of Electrophysiological Signals

This manuscript presents the design of a deep differential neural network (DDNN) for pattern classification. First, we proposed a DDNN topology with three layers, whose learning laws are derived from a Lyapunov analysis, justifying local asymptotic convergence of the classification error and the weights of the DDNN. Then, an extension to include an arbitrary number of hidden layers in the DDNN is analyzed. The learning laws for this general form of the DDNN offer a contribution to the deep learning framework for signal classification with biological nature and dynamic structures. The DDNN is used to classify electroencephalographic signals from volunteers that perform an identification graphical test. The classification results show exponential growth in the signal classification accuracy from 82 percent with one layer to 100 percent with three hidden layers. Working with DDNN instead of static deep neural networks (SDNN) represents a set of advantages, such as processing time and training period reduction up to almost 100 times, and the increment of the classification accuracy while working with less hidden layers than working with SDNN, which are highly dependent on their topology and the number of neurons in each layer. The DDNN employed fewer neurons due to the induced feedback characteristic.

EPIC: Annotated epileptic EEG independent components for artifact reduction

Scalp electroencephalogram is a non-invasive multi-channel biosignal that records the brain’s electrical activity. It is highly susceptible to noise that might overshadow important data. Independent component analysis is one of the most used artifact removal methods. Independent component analysis separates data into different components, although it can not automatically reject the noisy ones. Therefore, experts are needed to decide which components must be removed before reconstructing the data. To automate this method, researchers have developed classifiers to identify noisy components. However, to build these classifiers, they need annotated data. Manually classifying independent components is a time-consuming task. Furthermore, few labelled data are publicly available. This paper presents a source of annotated electroencephalogram independent components acquired from patients with epilepsy (EPIC Dataset). This dataset contains 77,426 independent components obtained from approximately 613 hours of electroencephalogram, visually inspected by two experts, which was already successfully utilised to develop independent component classifiers.

Measurement(s)	Electroencephalogram Independent Components
Technology Type(s)	MATLAB and Python

An EEG-based systematic explainable detection framework for probing and localizing abnormal patterns in Alzheimer's disease

OBJECTIVE

Electroencephalography (EEG) is a potential source of downstream biomarkers for the early diagnosis of Alzheimer's disease (AD) due to its low-cost, non-invasive, and portable advantages. Accurately detecting AD-induced patterns from EEG signals is essential for understanding AD-related neurodegeneration at the EEG level and further evaluating the risk of AD at an early stage. This paper proposes a deep learning-based, functional explanatory framework that probes AD abnormalities from short-sequence EEG data.

APPROACH

The framework is a learning-based automatic detection system consisting of three encoding pathways that analyze EEG signals in frequency, complexity, and synchronous domains. We integrated the proposed EEG descriptors with the neural network components into one learning system to detect AD patterns. A transfer learning-based model was used to learn the deep representations, and a modified generative adversarial module was attached to the model to overcome feature sparsity. Furthermore, we utilized activation mapping to obtain the AD-related neurodegeneration at brain rhythm, dynamic complexity, and functional connectivity levels.

MAIN RESULTS

The proposed framework can accurately (100%) detect AD patterns based on our raw EEG recordings without delicate preprocessing. Meanwhile, the system indicates that 1) the power of different brain rhythms exhibits abnormal in the frontal lobes of AD patients, and such abnormality spreads to central lobes in the alpha and beta rhythms, 2) the difference in nonlinear complexity varies with the temporal scales, and 3) all the connections of pair-wise brain regions except bilateral temporal connectivity are weak in AD patterns. The proposed method outperforms other related methods in detection performance.

SIGNIFICANCE

We provide a new method for revealing abnormalities and corresponding localizations in different feature domains of EEG from AD patients. This study is a significant foundation for our future work on identifying individuals at high risk of AD at an early stage.

Ensemble Deep Neural Network for Automatic Classification of EEG Independent Components

OBJECTIVE

Independent component analysis (ICA) is commonly used to remove noisy artifacts from multi-channel scalp electroencephalogram (EEG) signals. ICA decomposes EEG into different independent components (ICs) and then, experts remove the noisy ones. This process is highly time-consuming and experts are not always available. To surpass this drawback, research is going on to develop models to automatically conduct IC classification. Current state-of-the-art models use power spectrum densities (PSDs) and topoplots to classify ICs. The performance of these methods may be limited by disregarding the IC time-series that would contribute to fully simulate the visual inspection performed by experts.

METHODS

We present a novel ensemble deep neural network that combines time-series, PSDs, and topoplots to classify ICs. Moreover, we study the ability to use our model in transfer learning approaches.

RESULTS

Experimental results showed that using time-series improves IC classification. Results also indicated that transfer learning obtained higher performance than simply training a new model from scratch.

CONCLUSION

Researchers should develop IC classifiers using the three sources of information. Moreover, transfer learning approaches should be considered when producing new deep learning models.

SIGNIFICANCE

This work improves IC classification, enhancing the automatic removal of EEG artifacts. Additionally, since labelled ICs are generally not publicly available, the possibility of using our model in transfer learning studies may motivate other researchers to develop their own classifiers.

Inhibition of return in time-lapse: Brain Rhythms during grip force control for spatial attention

The inhibition of return (IoR) is the observable slowed response to a target at a cued position for cue-target intervals of longer than 300 ms; when there has been enough time to disengage from a previously-cued location, an inhibitory after-effect can be observed. Studies aimed at understanding whether mechanisms underlying IoR act at a perceptual/attentional or a later response-execution stage have offered divergent results. Though focusing on the brain's responses to cue-target intervals can offer significant information on the nature of IoR, few studies have investigated neural activity during this interval; these studies suggest the generation of inhibitory tags on the spatial coordinates of the previously attended position which, in turn, inhibit motor programming toward that position. As such, a cue-target task was administered in this study; the rhythmic activity of EEG signals on the entire cue-target interval was measured to determine whether IoR is referred to early or late response processing stages. A visually-guided force variation during isometric contraction, instead of a key press response, was required to reduce the effect of motor response initiation. Our results indicated the prominent involvement of the fronto-parietal and occipital cortical areas post-cue appearance, with a peculiar theta band modulation characterizing the posterior parietal cortex. Theta activity in this region was enhanced post-cue onset, decreased over time, and was enhanced again when a target appeared in an unexpected location rather than in a cued position. This suggests that the mechanism that generates IoR sequentially affects perceptual/attentional processing and motor preparation rather than response execution.

Sensory-Motor Modulations of EEG Event-Related Potentials Reflect Walking-Related Macro-Affordances

One fundamental principle of the brain functional organization is the elaboration of sensory information for the specification of action plans that are most appropriate for interaction with the environment. Using an incidental go/no-go priming paradigm, we have previously shown a facilitation effect for the execution of a walking-related action in response to far vs. near objects/locations in the extrapersonal space, and this effect has been called “macro-affordance” to reflect the role of locomotion in the coverage of extrapersonal distance. Here, we investigated the neurophysiological underpinnings of such an effect by recording scalp electroencephalography (EEG) from 30 human participants during the same paradigm. The results of a whole-brain analysis indicated a significant modulation of the event-related potentials (ERPs) both during prime and target stimulus presentation. Specifically, consistent with a mechanism of action anticipation and automatic activation of affordances, a stronger ERP was observed in response to prime images framing the environment from a far vs. near distance, and this modulation was localized in dorso-medial motor regions. In addition, an inversion of polarity for far vs. near conditions was observed during the subsequent target period in dorso-medial parietal regions associated with spatially directed foot-related actions. These findings were interpreted within the framework of embodied models of brain functioning as arising from a mechanism of motor-anticipation and subsequent prediction error which was guided by the preferential affordance relationship between the distant large-scale environment and locomotion. More in general, our findings reveal a sensory-motor mechanism for the processing of walking-related environmental affordances.

MEGnet: Automatic ICA-based artifact removal for MEG using spatiotemporal convolutional neural networks

Magnetoencephalography (MEG) is a functional neuroimaging tool that records the magnetic fields induced by neuronal activity; however, signal from non-neuronal sources can corrupt the data. Eye-blinks, saccades, and cardiac activity are three of the most common sources of non-neuronal artifacts. They can be measured by affixing eye proximal electrodes, as in electrooculography (EOG), and chest electrodes, as in electrocardiography (ECG), however this complicates imaging setup, decreases patient comfort, and can induce further artifacts from movement. This work proposes an EOG- and ECG-free approach to identify eye-blinks, saccades, and cardiac activity signals for automated artifact suppression. The contribution of this work is three-fold. First, using a data driven, multivariate decomposition approach based on Independent Component Analysis (ICA), a highly accurate artifact classifier is constructed as an amalgam of deep 1-D and 2-D Convolutional Neural Networks (CNNs) to automate the identification and removal of ubiquitous whole brain artifacts including eye-blink, saccade, and cardiac artifacts. The specific architecture of this network is optimized through an unbiased, computer-based hyperparameter random search. Second, visualization methods are applied to the learned abstraction to reveal what features the model uses and to bolster user confidence in the model's training and potential for generalization. Finally, the model is trained and tested on both resting-state and task MEG data from 217 subjects, and achieves a new state-of-the-art in artifact detection accuracy of 98.95% including 96.74% sensitivity and 99.34% specificity on the held out test-set. This work automates MEG processing for both clinical and research use, adapts to the acquired acquisition time, and can obviate the need for EOG or ECG electrodes for artifact detection.

Machine learning for predicting levetiracetam treatment response in temporal lobe epilepsy

OBJECTIVE

To determine the predictive power for seizure-freedom of 19-channels EEG, measured both before and after three months the initiation of the use of Levetiracetam (LEV), in a cohort of people after a new diagnosis of temporal-lobe epilepsy (TLE) using a machine-learning approach.

METHODS

Twenty-three individuals with TLE were examined. We dichotomized clinical outcome into seizure-free (SF) and non-seizure-free (NSF) after two years of LEV. EEG effective power in different frequency bands was compared using baseline EEG (T0) and the EEG after three months of LEV therapy (T1) between SF and NSF patients. Partial Least Square (PLS) analysis was used to test and validate the prediction of the model for clinical outcome.

RESULTS

A total of 152 features were extracted from the EEG recordings. When considering only the features calculated at T1, a predictive power for seizure-freedom (AUC = 0.750) was obtained. When employing both T0 and T1 features, an AUC = 0.800 was obtained.

CONCLUSIONS

This study provides a proof-of-concept pipeline for predicting the clinical response to anti-seizure medications in people with epilepsy.

SIGNIFICANCE

Future studies may benefit from the pipeline proposed in this study in order to develop a model that can match each patient to the most effective anti-seizure medication.

Application of bi-modal signal in the classification and recognition of drug addiction degree based on machine learning

Most studies on drug addiction degree are made based on statistical scales, addicts' account, and subjective judgement of rehabilitation doctors. No objective, quantified evaluation has been made. This paper uses devises the synchronous bimodal signal collection and experimentation paradigm with electroencephalogram (EEG) and forehead high-density near-infrared spectroscopy (NIRS) device. The drug addicts are classified into mild, moderate and severe groups with reference to the suggestions of researchers and medical experts. Data of 45 drug addicts (mild: 15; moderate: 15; and severe: 15) is collected, and then used to design an addiction degree testing algorithm based on decision fusion. The algorithm is used to classify mild, moderate and severe addiction. This paper pioneers to use two types of Convolutional Neural Network (CNN) to abstract the EEG and NIR data of drug addicts, and introduces batch normalization to CNN, thus accelerating training process, reducing parameter sensitivity, and enhancing system robustness. The characteristics output by two CNNs are transformed into dimensions. Two new characteristics are assigned with a weight of 50% each. The data is used for decision fusion. In the networks, 27 subjects are used as training sets, 9 as validation sets, and 9 as testing sets. The 3-class accuracy remains to be 63.15%, preliminarily justifying this method as an effective approach to measure drug addiction degree. And the method is ready to use, objective, and offers results in real time.

Deep-MEG: spatiotemporal CNN features and multiband ensemble classification for predicting the early signs of Alzheimer’s disease with magnetoencephalography

In this paper, we present the novel Deep-MEG approach in which image-based representations of magnetoencephalography (MEG) data are combined with ensemble classifiers based on deep convolutional neural networks. For the scope of predicting the early signs of Alzheimer’s disease (AD), functional connectivity (FC) measures between the brain bio-magnetic signals originated from spatially separated brain regions are used as MEG data representations for the analysis. After stacking the FC indicators relative to different frequency bands into multiple images, a deep transfer learning model is used to extract different sets of deep features and to derive improved classification ensembles. The proposed Deep-MEG architectures were tested on a set of resting-state MEG recordings and their corresponding magnetic resonance imaging scans, from a longitudinal study involving 87 subjects. Accuracy values of 89% and 87% were obtained, respectively, for the early prediction of AD conversion in a sample of 54 mild cognitive impairment subjects and in a sample of 87 subjects, including 33 healthy controls. These results indicate that the proposed Deep-MEG approach is a powerful tool for detecting early alterations in the spectral–temporal connectivity profiles and in their spatial relationships.

Evidence of Neurovascular Un-Coupling in Mild Alzheimer's Disease through Multimodal EEG-fNIRS and Multivariate Analysis of Resting-State Data

Alzheimer’s disease (AD) is associated with modifications in cerebral blood perfusion and autoregulation. Hence, neurovascular coupling (NC) alteration could become a biomarker of the disease. NC might be assessed in clinical settings through multimodal electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). Multimodal EEG-fNIRS was recorded at rest in an ambulatory setting to assess NC and to evaluate the sensitivity and specificity of the methodology to AD. Global NC was evaluated with a general linear model (GLM) framework by regressing whole-head EEG power envelopes in three frequency bands (theta, alpha and beta) with average fNIRS oxy- and deoxy-hemoglobin concentration changes in the frontal and prefrontal cortices. NC was lower in AD compared to healthy controls (HC) with significant differences in the linkage of theta and alpha bands with oxy- and deoxy-hemoglobin, respectively (p = 0.028 and p = 0.020). Importantly, standalone EEG and fNIRS metrics did not highlight differences between AD and HC. Furthermore, a multivariate data-driven analysis of NC between the three frequency bands and the two hemoglobin species delivered a cross-validated classification performance of AD and HC with an Area Under the Curve, AUC = 0.905 (p = 2.17 × 10⁻⁵). The findings demonstrate that EEG-fNIRS may indeed represent a powerful ecological tool for clinical evaluation of NC and early identification of AD.

Is Brain Dynamics Preserved in the EEG After Automated Artifact Removal? A Validation of the Fingerprint Method and the Automatic Removal of Cardiac Interference Approach Based on Microstate Analysis

The assessment of a method for removing artifacts from electroencephalography (EEG) datasets often disregard verifying that global brain dynamics is preserved. In this study, we verified that the recently introduced optimized fingerprint method and the automatic removal of cardiac interference (ARCI) approach not only remove physiological artifacts from EEG recordings but also preserve global brain dynamics, as assessed with a new approach based on microstate analysis. We recorded EEG activity with a high-resolution EEG system during two resting-state conditions (eyes open, 25 volunteers, and eyes closed, 26 volunteers) known to exhibit different brain dynamics. After signal decomposition by independent component analysis (ICA), the independent components (ICs) related to eyeblinks, eye movements, myogenic interference, and cardiac electromechanical activity were identified with the optimized fingerprint method and ARCI approach and statistically compared with the outcome of the expert classification of the ICs by visual inspection. Brain dynamics in two different groups of denoised EEG signals, reconstructed after having removed the artifactual ICs identified by either visual inspection or the automated methods, was assessed by calculating microstate topographies, microstate metrics (duration, occurrence, and coverage), and directional predominance (based on transition probabilities). No statistically significant differences between the expert and the automated classification of the artifactual ICs were found (p > 0.05). Cronbach’s α values assessed the high test–retest reliability of microstate parameters for EEG datasets denoised by the automated procedure. The total EEG signal variance explained by the sets of global microstate templates was about 80% for all denoised EEG datasets, with no significant differences between groups. For the differently denoised EEG datasets in the two recording conditions, we found that the global microstate templates and the sequences of global microstates were very similar (p < 0.01). Descriptive statistics and Cronbach’s α of microstate metrics highlighted no significant differences and excellent consistency between groups (p > 0.5). These results confirm the ability of the optimized fingerprint method and the ARCI approach to effectively remove physiological artifacts from EEG recordings while preserving global brain dynamics. They also suggest that microstate analysis could represent a novel approach for assessing the ability of an EEG denoising method to remove artifacts without altering brain dynamics.

EEG-Based Estimation on the Reduction of Negative Emotions for Illustrated Surgical Images

Electroencephalogram (EEG) biosignals are widely used to measure human emotional reactions. The recent progress of deep learning-based classification models has improved the accuracy of emotion recognition in EEG signals. We apply a deep learning-based emotion recognition model from EEG biosignals to prove that illustrated surgical images reduce the negative emotional reactions that the photographic surgical images generate. The strong negative emotional reactions caused by surgical images, which show the internal structure of the human body (including blood, flesh, muscle, fatty tissue, and bone) act as an obstacle in explaining the images to patients or communicating with the images with non-professional people. We claim that the negative emotional reactions generated by illustrated surgical images are less severe than those caused by raw surgical images. To demonstrate the difference in emotional reaction, we produce several illustrated surgical images from photographs and measure the emotional reactions they engender using EEG biosignals; a deep learning-based emotion recognition model is applied to extract emotional reactions. Through this experiment, we show that the negative emotional reactions associated with photographic surgical images are much higher than those caused by illustrated versions of identical images. We further execute a self-assessed user survey to prove that the emotions recognized from EEG signals effectively represent user-annotated emotions.

Working Memory Decline in Alzheimer's Disease Is Detected by Complexity Analysis of Multimodal EEG-fNIRS

Alzheimer's disease (AD) is characterized by working memory (WM) failures that can be assessed at early stages through administering clinical tests. Ecological neuroimaging, such as Electroencephalography (EEG) and functional Near Infrared Spectroscopy (fNIRS), may be employed during these tests to support AD early diagnosis within clinical settings. Multimodal EEG-fNIRS could measure brain activity along with neurovascular coupling (NC) and detect their modifications associated with AD. Data analysis procedures based on signal complexity are suitable to estimate electrical and hemodynamic brain activity or their mutual information (NC) during non-structured experimental paradigms. In this study, sample entropy of whole-head EEG and frontal/prefrontal cortex fNIRS was evaluated to assess brain activity in early AD and healthy controls (HC) during WM tasks (i.e., Rey-Osterrieth complex figure and Raven's progressive matrices). Moreover, conditional entropy between EEG and fNIRS was evaluated as indicative of NC. The findings demonstrated the capability of complexity analysis of multimodal EEG-fNIRS to detect WM decline in AD. Furthermore, a multivariate data-driven analysis, performed on these entropy metrics and based on the General Linear Model, allowed classifying AD and HC with an AUC up to 0.88. EEG-fNIRS may represent a powerful tool for the clinical evaluation of WM decline in early AD.

Electroencephalography-Derived Prognosis of Functional Recovery in Acute Stroke Through Machine Learning Approaches

Stroke, if not lethal, is a primary cause of disability. Early assessment of markers of recovery can allow personalized interventions; however, it is difficult to deliver indexes in the acute phase able to predict recovery. In this perspective, evaluation of electrical brain activity may provide useful information. A machine learning approach was explored here to predict post-stroke recovery relying on multi-channel electroencephalographic (EEG) recordings of few minutes performed at rest. A data-driven model, based on partial least square (PLS) regression, was trained on 19-channel EEG recordings performed within 10 days after mono-hemispheric stroke in 101 patients. The band-wise (delta: 1-4[Formula: see text]Hz, theta: 4-7[Formula: see text]Hz, alpha: 8-14[Formula: see text]Hz and beta: 15-30[Formula: see text]Hz) EEG effective powers were used as features to predict the recovery at 6 months (based on clinical status evaluated through the NIH Stroke Scale, NIHSS) in an optimized and cross-validated framework. In order to exploit the multimodal contribution to prognosis, the EEG-based prediction of recovery was combined with NIHSS scores in the acute phase and both were fed to a nonlinear support vector regressor (SVR). The prediction performance of EEG was at least as good as that of the acute clinical status scores. A posteriori evaluation of the features exploited by the analysis highlighted a lower delta and higher alpha activity in patients showing a positive outcome, independently of the affected hemisphere. The multimodal approach showed better prediction capabilities compared to the acute NIHSS scores alone ([Formula: see text] versus [Formula: see text], AUC = 0.80 versus AUC = 0.70, [Formula: see text]). The multimodal and multivariate model can be used in acute phase to infer recovery relying on standard EEG recordings of few minutes performed at rest together with clinical assessment, to be exploited for early and personalized therapies. The easiness of performing EEG may allow such an approach to become a standard-of-care and, thanks to the increasing number of labeled samples, further improving the model predictive power.

Thinker invariance: enabling deep neural networks for BCI across more people

OBJECTIVE

Most deep neural networks (DNNs) used as brain computer interfaces (BCI) classifiers are rarely viable for more than one person and are relatively shallow compared to the state-of-the-art in the wider machine learning literature. The goal of this work is to frame these as a unified challenge and reconsider how transfer learning is used to overcome these difficulties.

APPROACH

We present two variations of a holistic approach to transfer learning with DNNs for BCI that rely on a deeper network called TIDNet. Our approaches use multiple subjects for training in the interest of creating a more universal classifier that is applicable for new (unseen) subjects. The first approach is purely subject-invariant and the second targets specific subjects, without loss of generality. We use five publicly accessible datasets covering a range of tasks and compare our approaches to state-of-the-art alternatives in detail.

MAIN RESULTS

We observe that TIDNet in conjunction with our training augmentations is more consistent when compared to shallower baselines, and in some cases exhibits large and significant improvements, for instance motor imagery classification improvements of over 8%. Furthermore, we show that our suggested multi-domain learning (MDL) strategy strongly outperforms simply fine-tuned general models when targeting specific subjects, while remaining more generalizable to still unseen subjects.

SIGNIFICANCE

TIDNet in combination with a data alignment-based training augmentation proves to be a consistent classification approach of single raw trials and can be trained even with the inclusion of corrupted trials. Our MDL strategy calls into question the intuition to fine-tune trained classifiers to new subjects, as it proves simpler and more accurate while remaining general. Furthermore, we show evidence that augmented TIDNet training makes better use of additional subjects, showing continued and greater performance improvement over shallower alternatives, indicating promise for a new subject-invariant paradigm rather than a subject-specific one.

Transcutaneous Vagus Nerve Stimulation Modulates EEG Microstates and Delta Activity in Healthy Subjects

Transcutaneous vagus nerve stimulation (tVNS) is an alternative non-invasive method for the electrical stimulation of the vagus nerve with the goal of treating several neuropsychiatric disorders. The objective of this study is to assess the effects of tVNS on cerebral cortex activity in healthy volunteers using resting-state microstates and power spectrum electroencephalography (EEG) analysis. Eight male subjects aged 25–45 years were recruited in this randomized sham-controlled double-blind study with cross-over design. Real tVNS was administered at the left external acoustic meatus, while sham stimulation was performed at the left ear lobe, both of them for 60 min. The EEG recording lasted 5 min and was performed before and 60 min following the tVNS experimental session. We observed that real tVNS induced an increase in the metrics of microstate A mean duration (p = 0.039) and an increase in EEG power spectrum activity in the delta frequency band (p < 0.01). This study confirms that tVNS is an effective way to stimulate the vagus nerve, and the mechanisms of action of this activation can be successfully studied using scalp EEG quantitative metrics. Future studies are warranted to explore the clinical implications of these findings and to focus the research of the prognostic biomarkers of tVNS therapy for neuropsychiatric diseases.

Emotion Variation from Controlling Contrast of Visual Contents through EEG-Based Deep Emotion Recognition

Visual contents such as movies and animation evoke various human emotions. We examine an argument that the emotion from the visual contents may vary according to the contrast control of the scenes contained in the contents. We sample three emotions including positive, neutral and negative to prove our argument. We also sample several scenes of these emotions from visual contents and control the contrast of the scenes. We manipulate the contrast of the scenes and measure the change of valence and arousal from human participants who watch the contents using a deep emotion recognition module based on electroencephalography (EEG) signals. As a result, we conclude that the enhancement of contrast induces the increase of valence, while the reduction of contrast induces the decrease. Meanwhile, the contrast control affects arousal on a very minute scale.

EEG Artifact Removal by Bayesian Deep Learning & ICA

Artifact removal is important for EEG signal processing because artifacts adversely affect analysis results. To preserve normal EEG signal, several methods based on independent component analysis (ICA) have been studied and artifacts are removed by discarding independent components (ICs) classified as artifacts. In this study, a method using Bayesian deep learning and attention module is presented to improve performance of the classifier for ICs. Probability value is computed through the method to predict if a component is artifact and to treat ambiguous inputs. The attention module achieves increasing classification accuracy and shows the map of the region where the classifier concentrates on.

Euthymic bipolar disorder patients and EEG microstates: a neural signature of their abnormal self experience?

BACKGROUND

A growing number of neuroimaging studies have revealed spatial abnormalities of resting-state functional brain network activity in bipolar disorder (BD). Conversely, abnormalities of resting state temporal dynamics have been scarcely investigated so far. The aim of this study was to characterize the EEG microstates activity in BD patients with a history of manic predominant polarity. Patients were euthymic and pharmacologically stabilized.

METHODS

Nineteen BD patients (mean age 34.4 ± 11.0, 7 female) and 19 healthy controls (HC; mean age 38.2 ± 9.9, 7 female) were recruited. The psychometric evaluation included the Hamilton Depression Scale (HAMD), the Young Mania Rating Scale (YMRS), the Dissociative Experience Scale (DES), and the State-Trait Anxiety Inventory (STAI).  Two runs of 2 minutes of EEG activity by a 128-channel system were acquired at rest and analyzed through microstate analysis.

RESULTS

We found a reduced presence of microstate B in BD patients compared to HC, since BD patients have a tendency to transit from the microstate B to the microstates C and D significantly more than HC. Furthermore, microstate B features were correlated with DES, state STAI and trait STAI scores.

CONCLUSION

The reduced presence of microstate B might be associated with episodic autobiographic memory deficit, exaggerated self-focusing and states of dissociations characteristic of BD. Strong correlations of microstate B metrics and dynamics with symptoms of dissociation and anxiety across the two groups supported this interpretation.

Fiberless, Multi-Channel fNIRS-EEG System Based on Silicon Photomultipliers: Towards Sensitive and Ecological Mapping of Brain Activity and Neurovascular Coupling

Portable neuroimaging technologies can be employed for long-term monitoring of neurophysiological and neuropathological states. Functional Near-Infrared Spectroscopy (fNIRS) and Electroencephalography (EEG) are highly suited for such a purpose. Their multimodal integration allows the evaluation of hemodynamic and electrical brain activity together with neurovascular coupling. An innovative fNIRS-EEG system is here presented. The system integrated a novel continuous-wave fNIRS component and a modified commercial EEG device. fNIRS probing relied on fiberless technology based on light emitting diodes and silicon photomultipliers (SiPMs). SiPMs are sensitive semiconductor detectors, whose large detection area maximizes photon harvesting from the scalp and overcomes limitations of fiberless technology. To optimize the signal-to-noise ratio and avoid fNIRS-EEG interference, a digital lock-in was implemented for fNIRS signal acquisition. A benchtop characterization of the fNIRS component showed its high performances with a noise equivalent power below 1 pW. Moreover, the fNIRS-EEG device was tested in vivo during tasks stimulating visual, motor and pre-frontal cortices. Finally, the capabilities to perform ecological recordings were assessed in clinical settings on one Alzheimer's Disease patient during long-lasting cognitive tests. The system can pave the way to portable technologies for accurate evaluation of multimodal brain activity, allowing their extensive employment in ecological environments and clinical practice.

Thermal Infrared Imaging-Based Affective Computing and Its Application to Facilitate Human Robot Interaction: A Review

Over recent years, robots are increasingly being employed in several aspects of modern society. Among others, social robots have the potential to benefit education, healthcare, and tourism. To achieve this purpose, robots should be able to engage humans, recognize users’ emotions, and to some extent properly react and "behave" in a natural interaction. Most robotics applications primarily use visual information for emotion recognition, which is often based on facial expressions. However, the display of emotional states through facial expression is inherently a voluntary controlled process that is typical of human–human interaction. In fact, humans have not yet learned to use this channel when communicating with a robotic technology. Hence, there is an urgent need to exploit emotion information channels not directly controlled by humans, such as those that can be ascribed to physiological modulations. Thermal infrared imaging-based affective computing has the potential to be the solution to such an issue. It is a validated technology that allows the non-obtrusive monitoring of physiological parameters and from which it might be possible to infer affective states. This review is aimed to outline the advantages and the current research challenges of thermal imaging-based affective computing for human–robot interaction.

Mammography Image Quality Assurance Using Deep Learning

OBJECTIVE

According to the European Reference Organization for Quality Assured Breast Cancer Screening and Diagnostic Services (EUREF) image quality in mammography is assessed by recording and analyzing a set of images of the CDMAM phantom. The EUREF procedure applies an automated analysis combining image registration, signal detection and nonlinear fitting. We present a proof of concept for an end-to-end deep learning framework that assesses image quality on the basis of single images as an alternative.

METHODS

Virtual mammography is used to generate a database with known ground truth for training a regression convolutional neural net (CNN). Training is carried out by continuously extending the training data and applying transfer learning.

RESULTS

The trained net is shown to correctly predict the image quality of simulated and real images. Specifically, image quality predictions on the basis of single images are of similar quality as those obtained by applying the EUREF procedure with 16 images. Our results suggest that the trained CNN generalizes well.

CONCLUSION

Mammography image quality assessment can benefit from the proposed deep learning approach.

SIGNIFICANCE

Deep learning avoids cumbersome pre-processing and allows mammography image quality to be estimated reliably using single images.

Deep Learning in Physiological Signal Data: A Survey

Deep Learning (DL), a successful promising approach for discriminative and generative tasks, has recently proved its high potential in 2D medical imaging analysis; however, physiological data in the form of 1D signals have yet to be beneficially exploited from this novel approach to fulfil the desired medical tasks. Therefore, in this paper we survey the latest scientific research on deep learning in physiological signal data such as electromyogram (EMG), electrocardiogram (ECG), electroencephalogram (EEG), and electrooculogram (EOG). We found 147 papers published between January 2018 and October 2019 inclusive from various journals and publishers. The objective of this paper is to conduct a detailed study to comprehend, categorize, and compare the key parameters of the deep-learning approaches that have been used in physiological signal analysis for various medical applications. The key parameters of deep-learning approach that we review are the input data type, deep-learning task, deep-learning model, training architecture, and dataset sources. Those are the main key parameters that affect system performance. We taxonomize the research works using deep-learning method in physiological signal analysis based on: (1) physiological signal data perspective, such as data modality and medical application; and (2) deep-learning concept perspective such as training architecture and dataset sources.

Distinguishing Emotional Responses to Photographs and Artwork Using a Deep Learning-Based Approach

Visual stimuli from photographs and artworks raise corresponding emotional responses. It is a long process to prove whether the emotions that arise from photographs and artworks are different or not. We answer this question by employing electroencephalogram (EEG)-based biosignals and a deep convolutional neural network (CNN)-based emotion recognition model. We employ Russell's emotion model, which matches emotion keywords such as happy, calm or sad to a coordinate system whose axes are valence and arousal, respectively. We collect photographs and artwork images that match the emotion keywords and build eighteen one-minute video clips for nine emotion keywords for photographs and artwork. We hired forty subjects and executed tests about the emotional responses from the video clips. From the t-test on the results, we concluded that the valence shows difference, while the arousal does not.

Exploring Shopper's Browsing Behavior and Attention Level with an EEG Biosensor Cap

The online shopping market is developing rapidly, meaning that it is important for retailers and manufacturers to understand how consumers behave online compared to when in brick-and-mortar stores. Retailers want consumers to spend time shopping, browsing, and searching for products in the hope a purchase is made. On the other hand, consumers may want to restrict their duration of stay on websites due to perceived risk of loss of time or convenience. This phenomenon underlies the need to reduce the duration of consumer stay (namely, time pressure) on websites. In this paper, the browsing behavior and attention span of shoppers engaging in online shopping under time pressure were investigated. The attention and meditation level are measured by an electroencephalogram (EEG) biosensor cap. The results indicated that when under time pressure shoppers engaging in online shopping are less attentive. Thus, marketers may need to find strategies to increase a shopper’s attention. Shoppers unfamiliar with product catalogs on shopping websites are less attentive, therefore marketers should adopt an interesting style for product catalogs to hold a shopper’s attention. We discuss our findings and outline their business implications.