Convolutional neural networks for decoding electroencephalography responses and visualizing trial by trial changes in discriminant features

An empirical comparison of deep learning explainability approaches for EEG using simulated ground truth

Recent advancements in machine learning and deep learning (DL) based neural decoders have significantly improved decoding capabilities using scalp electroencephalography (EEG). However, the interpretability of DL models remains an under-explored area. In this study, we compared multiple model explanation methods to identify the most suitable method for EEG and understand when some of these approaches might fail. A simulation framework was developed to evaluate the robustness and sensitivity of twelve back-propagation-based visualization methods by comparing to ground truth features. Multiple methods tested here showed reliability issues after randomizing either model weights or labels: e.g., the saliency approach, which is the most used visualization technique in EEG, was not class or model-specific. We found that DeepLift was consistently accurate as well as robust to detect the three key attributes tested here (temporal, spatial, and spectral precision). Overall, this study provides a review of model explanation methods for DL-based neural decoders and recommendations to understand when some of these methods fail and what they can capture in EEG.

Survey on the research direction of EEG-based signal processing

Electroencephalography (EEG) is increasingly important in Brain-Computer Interface (BCI) systems due to its portability and simplicity. In this paper, we provide a comprehensive review of research on EEG signal processing techniques since 2021, with a focus on preprocessing, feature extraction, and classification methods. We analyzed 61 research articles retrieved from academic search engines, including CNKI, PubMed, Nature, IEEE Xplore, and Science Direct. For preprocessing, we focus on innovatively proposed preprocessing methods, channel selection, and data augmentation. Data augmentation is classified into conventional methods (sliding windows, segmentation and recombination, and noise injection) and deep learning methods [Generative Adversarial Networks (GAN) and Variation AutoEncoder (VAE)]. We also pay attention to the application of deep learning, and multi-method fusion approaches, including both conventional algorithm fusion and fusion between conventional algorithms and deep learning. Our analysis identifies 35 (57.4%), 18 (29.5%), and 37 (60.7%) studies in the directions of preprocessing, feature extraction, and classification, respectively. We find that preprocessing methods have become widely used in EEG classification (96.7% of reviewed papers) and comparative experiments have been conducted in some studies to validate preprocessing. We also discussed the adoption of channel selection and data augmentation and concluded several mentionable matters about data augmentation. Furthermore, deep learning methods have shown great promise in EEG classification, with Convolutional Neural Networks (CNNs) being the main structure of deep neural networks (92.3% of deep learning papers). We summarize and analyze several innovative neural networks, including CNNs and multi-structure fusion. However, we also identified several problems and limitations of current deep learning techniques in EEG classification, including inappropriate input, low cross-subject accuracy, unbalanced between parameters and time costs, and a lack of interpretability. Finally, we highlight the emerging trend of multi-method fusion approaches (49.2% of reviewed papers) and analyze the data and some examples. We also provide insights into some challenges of multi-method fusion. Our review lays a foundation for future studies to improve EEG classification performance.

Auditory stimulation and deep learning predict awakening from coma after cardiac arrest

Assessing the integrity of neural functions in coma after cardiac arrest remains an open challenge. Prognostication of coma outcome relies mainly on visual expert scoring of physiological signals, which is prone to subjectivity and leaves a considerable number of patients in a 'grey zone', with uncertain prognosis. Quantitative analysis of EEG responses to auditory stimuli can provide a window into neural functions in coma and information about patients' chances of awakening. However, responses to standardized auditory stimulation are far from being used in a clinical routine due to heterogeneous and cumbersome protocols. Here, we hypothesize that convolutional neural networks can assist in extracting interpretable patterns of EEG responses to auditory stimuli during the first day of coma that are predictive of patients' chances of awakening and survival at 3 months. We used convolutional neural networks (CNNs) to model single-trial EEG responses to auditory stimuli in the first day of coma, under standardized sedation and targeted temperature management, in a multicentre and multiprotocol patient cohort and predict outcome at 3 months. The use of CNNs resulted in a positive predictive power for predicting awakening of 0.83 ± 0.04 and 0.81 ± 0.06 and an area under the curve in predicting outcome of 0.69 ± 0.05 and 0.70 ± 0.05, for patients undergoing therapeutic hypothermia and normothermia, respectively. These results also persisted in a subset of patients that were in a clinical 'grey zone'. The network's confidence in predicting outcome was based on interpretable features: it strongly correlated to the neural synchrony and complexity of EEG responses and was modulated by independent clinical evaluations, such as the EEG reactivity, background burst-suppression or motor responses. Our results highlight the strong potential of interpretable deep learning algorithms in combination with auditory stimulation to improve prognostication of coma outcome.

The clinical feasibility of artificial intelligence-assisted compressed sensing single-shot fluid-attenuated inversion recovery (ACS-SS-FLAIR) for evaluation of uncooperative patients with brain diseases: comparison with the conventional T2-FLAIR with parallel imaging

Background

Satisfactory magnetic resonance imaging (MRI) of those patients with involuntary head motion due to brain diseases is essential in avoiding missed diagnosis and guiding treatment.

Purpose

To investigate the clinical feasibility of artificial intelligence-assisted compressed sensing single-shot fluid-attenuated inversion recovery (ACS-SS-FLAIR) in evaluating patients with involuntary head motion due to brain diseases, compared with the conventional T2-FLAIR with parallel imaging (PI-FLAIR).

Material and Methods

A total of 33 uncooperative patients with brain disease were prospectively enrolled. Two readers independently reviewed images acquired with ACS-SS-FLAIR and PI-FLAIR at a 3.0-T MR scanner. In the aspects of qualitative evaluation of image quality, overall image quality and lesion conspicuity of ACS-SS-FLAIR and PI-FLAIR were assessed and then statistically compared by paired Wilcoxon rank-sum test. For quantitative evaluation, the relative contrast of lesion-to-cerebral parenchyma were calculated and compared.

Results

Overall image quality scores of ACS-SS-FLAIR evaluated by two readers were 2.94 ± 0.24 and 2.91 ± 0.29, respectively, both of which were significantly higher than that of PI-FLAIR, respectively ( P < 0.001 and <0.001). Lesion conspicuity scores of were 2.74 ± 0.47 and 2.79 ± 0.44, both of which were significantly higher than that of PI-FLAIR, respectively ( P < 0.001 and <0.001). In the quantitative evaluation for image quality, the relative contrast of lesion-to-cerebral parenchyma was 0.34 ± 0.09 in the ACS-SS-FLAIR sequence, significantly larger than that in the PI-FLAIR sequence ( P = 0.001).

Conclusion

The ACS-SS-FLAIR sequence is clinically feasible in the MRI workup of those patients with involuntary head motion due to brain diseases, showing shorter image acquisition time and better image quality compared with conventional PI-FLAIR.

Diagnostic and prognostic EEG analysis of critically ill patients: A deep learning study

Visual interpretation of electroencephalography (EEG) is time consuming, may lack objectivity, and is restricted to features detectable by a human. Computer-based approaches, especially deep learning, could potentially overcome these limitations. However, most deep learning studies focus on a specific question or a single pathology. Here we explore the potential of deep learning for EEG-based diagnostic and prognostic assessment of patients with acute consciousness impairment (ACI) of various etiologies. EEGs from 358 adults from a randomized controlled trial (CERTA, NCT03129438) were retrospectively analyzed. A convolutional neural network was used to predict the clinical outcome (based either on survival or on best cerebral performance category) and to determine the etiology (four diagnostic categories). The largest probability output served as marker for the confidence of the network in its prediction ("certainty factor"); we also systematically compared the predictions with raw EEG data, and used a visualization algorithm (Grad-CAM) to highlight discriminative patterns. When all patients were considered, the area under the receiver operating characteristic curve (AUC) was 0.721 for predicting survival and 0.703 for predicting the outcome based on best CPC; for patients with certainty factor ≥ 60 % the AUCs increased to 0.776 and 0.755 respectively; and for certainty factor ≥ 75 % to 0.852 and 0.879. The accuracy for predicting the etiology was 54.5 %; the accuracy increased to 67.7 %, 70.3 % and 84.1 % for patients with certainty factor of 50 %, 60 % and 75 % respectively. Visual analysis showed that the network learnt EEG patterns typically recognized by human experts, and suggested new criteria. This work demonstrates for the first time the potential of deep learning-based EEG analysis in critically ill patients with various etiologies of ACI. Certainty factor and post-hoc correlation of input data with prediction help to better characterize the method and pave the route for future implementations in clinical routine.

Image-Based Learning Using Gradient Class Activation Maps for Enhanced Physiological Interpretability of Motor Imagery Skills

Brain activity stimulated by the motor imagery paradigm (MI) is measured by Electroencephalography (EEG), which has several advantages to be implemented with the widely used Brain–Computer Interfaces (BCIs) technology. However, the substantial inter/intra variability of recorded data significantly influences individual skills on the achieved performance. This study explores the ability to distinguish between MI tasks and the interpretability of the brain’s ability to produce elicited mental responses with improved accuracy. We develop a Deep and Wide Convolutional Neuronal Network fed by a set of topoplots extracted from the multichannel EEG data. Further, we perform a visualization technique based on gradient-based class activation maps (namely, GradCam++) at different intervals along the MI paradigm timeline to account for intra-subject variability in neural responses over time. We also cluster the dynamic spatial representation of the extracted maps across the subject set to come to a deeper understanding of MI-BCI coordination skills. According to the results obtained from the evaluated GigaScience Database of motor-evoked potentials, the developed approach enhances the physiological explanation of motor imagery in aspects such as neural synchronization between rhythms, brain lateralization, and the ability to predict the MI onset responses and their evolution during training sessions.