Single-trial extraction of event-related potentials (ERPs) and classification of visual stimuli by ensemble use of discrete wavelet transform with Huffman coding and machine learning techniques

BACKGROUND

Presentation of visual stimuli can induce changes in EEG signals that are typically detectable by averaging together data from multiple trials for individual participant analysis as well as for groups or conditions analysis of multiple participants. This study proposes a new method based on the discrete wavelet transform with Huffman coding and machine learning for single-trial analysis of evenal (ERPs) and classification of different visual events in the visual object detection task.

METHODS

EEG single trials are decomposed with discrete wavelet transform (DWT) up to the [Formula: see text] level of decomposition using a biorthogonal B-spline wavelet. The coefficients of DWT in each trial are thresholded to discard sparse wavelet coefficients, while the quality of the signal is well maintained. The remaining optimum coefficients in each trial are encoded into bitstreams using Huffman coding, and the codewords are represented as a feature of the ERP signal. The performance of this method is tested with real visual ERPs of sixty-eight subjects.

RESULTS

The proposed method significantly discards the spontaneous EEG activity, extracts the single-trial visual ERPs, represents the ERP waveform into a compact bitstream as a feature, and achieves promising results in classifying the visual objects with classification performance metrics: accuracies 93.60[Formula: see text], sensitivities 93.55[Formula: see text], specificities 94.85[Formula: see text], precisions 92.50[Formula: see text], and area under the curve (AUC) 0.93[Formula: see text] using SVM and k-NN machine learning classifiers.

CONCLUSION

The proposed method suggests that the joint use of discrete wavelet transform (DWT) with Huffman coding has the potential to efficiently extract ERPs from background EEG for studying evoked responses in single-trial ERPs and classifying visual stimuli. The proposed approach has O(N) time complexity and could be implemented in real-time systems, such as the brain-computer interface (BCI), where fast detection of mental events is desired to smoothly operate a machine with minds.

Exploring EEG Effective Connectivity Network in Estimating Influence of Color on Emotion and Memory

Color is a perceptual stimulus that has a significant impact on improving human emotion and memory. Studies have revealed that colored multimedia learning materials (MLMs) have a positive effect on learner's emotion and learning where it was assessed by subjective/objective measurements. This study aimed to quantitatively assess the influence of colored MLMs on emotion, cognitive processes during learning, and long-term memory (LTM) retention using electroencephalography (EEG). The dataset consisted of 45 healthy participants, and MLMs were designed in colored or achromatic illustrations to elicit emotion and that to assess its impact on LTM retention after 30-min and 1-month delay. The EEG signal analysis was first started to estimate the effective connectivity network (ECN) using the phase slope index and expand it to characterize the ECN pattern using graph theoretical analysis. EEG results showed that colored MLMs had influences on theta and alpha networks, including (1) an increased frontal-parietal connectivity (top-down processing), (2) a larger number of brain hubs, (3) a lower clustering coefficient, and (4) a higher local efficiency, indicating that color influences information processing in the brain, as reflected by ECN, together with a significant improvement in learner's emotion and memory performance. This is evidenced by a more positive emotional valence and higher recall accuracy for groups who learned with colored MLMs than that of achromatic MLMs. In conclusion, this paper demonstrated how the EEG ECN parameters could help quantify the influences of colored MLMs on emotion and cognitive processes during learning.

Neural correlates of eye contact in face-to-face verbal interaction: An EEG-based study of the extraversion personality trait

The extraversion personality trait has a positive correlation with social interaction. In neuroimaging studies, investigations on extraversion in face-to-face verbal interactions are still scarce. This study presents an electroencephalography (EEG)-based investigation of the extraversion personality trait in relation to eye contact during face-to-face interactions, as this is a vital signal in social interactions. A sample of healthy male participants were selected (consisting of sixteen more extraverted and sixteen less extraverted individuals) and evaluated with the Eysenck's Personality Inventory (EPI) and Big Five Inventory (BFI) tools. EEG alpha oscillations in the occipital region were measured to investigate extraversion personality trait correlates of eye contact during a face-to-face interaction task and an eyes-open condition. The results revealed that the extraversion personality trait has a significant positive correlation with EEG alpha coherence in the occipital region, presumably due to its relationship with eye contact during the interaction task. Furthermore, the decrease in EEG alpha power during the interaction task compared to the eyes-open condition was found to be greater in the less extraverted participants; however, no significant difference was observed between the less and more extraverted participants. Overall, these findings encourage further research towards the understanding of neural mechanism correlates of the extraversion personality trait-particularly in social interaction.

Classification of Visual and Non-visual Learners Using Electroencephalographic Alpha and Gamma Activities

This study analyzes the learning styles of subjects based on their electroencephalo-graphy (EEG) signals. The goal is to identify how the EEG features of a visual learner differ from those of a non-visual learner. The idea is to measure the students' EEGs during the resting states (eyes open and eyes closed conditions) and when performing learning tasks. For this purpose, 34 healthy subjects are recruited. The subjects have no background knowledge of the animated learning content. The subjects are shown the animated learning content in a video format. The experiment consists of two sessions and each session comprises two parts: (1) Learning task: the subjects are shown the animated learning content for an 8-10 min duration. (2) Memory retrieval task The EEG signals are measured during the leaning task and memory retrieval task in two sessions. The retention time for the first session was 30 min, and 2 months for the second session. The analysis is performed for the EEG measured during the memory retrieval tasks. The study characterizes and differentiates the visual learners from the non-visual learners considering the extracted EEG features, such as the power spectral density (PSD), power spectral entropy (PSE), and discrete wavelet transform (DWT). The PSD and DWT features are analyzed. The EEG PSD and DWT features are computed for the recorded EEG in the alpha and gamma frequency bands over 128 scalp sites. The alpha and gamma frequency band for frontal, occipital, and parietal regions are analyzed as these regions are activated during learning. The extracted PSD and DWT features are then reduced to 8 and 15 optimum features using principal component analysis (PCA). The optimum features are then used as an input to the k-nearest neighbor (k-NN) classifier using the Mahalanobis distance metric, with 10-fold cross validation and support vector machine (SVM) classifier using linear kernel, with 10-fold cross validation. The classification results showed 97% and 94% accuracies rate for the first session and 96% and 93% accuracies for the second session in the alpha and gamma bands for the visual learners and non-visual learners, respectively, for k-NN classifier for PSD features and 68% and 100% accuracies rate for first session and 100% accuracies rate for second session for DWT features using k-NN classifier for the second session in the alpha and gamma band. For PSD features 97% and 96% accuracies rate for the first session, 100% and 95% accuracies rate for second session using SVM classifier and 79% and 82% accuracy for first session and 56% and 74% accuracy for second session for DWT features using SVM classifier. The results showed that the PSDs in the alpha and gamma bands represent distinct and stable EEG signatures for visual learners and non-visual learners during the retrieval of the learned contents.

Classification of EEG Signals Based on Pattern Recognition Approach

Feature extraction is an important step in the process of electroencephalogram (EEG) signal classification. The authors propose a "pattern recognition" approach that discriminates EEG signals recorded during different cognitive conditions. Wavelet based feature extraction such as, multi-resolution decompositions into detailed and approximate coefficients as well as relative wavelet energy were computed. Extracted relative wavelet energy features were normalized to zero mean and unit variance and then optimized using Fisher's discriminant ratio (FDR) and principal component analysis (PCA). A high density EEG dataset validated the proposed method (128-channels) by identifying two classifications: (1) EEG signals recorded during complex cognitive tasks using Raven's Advance Progressive Metric (RAPM) test; (2) EEG signals recorded during a baseline task (eyes open). Classifiers such as, K-nearest neighbors (KNN), Support Vector Machine (SVM), Multi-layer Perceptron (MLP), and Naïve Bayes (NB) were then employed. Outcomes yielded 99.11% accuracy via SVM classifier for coefficient approximations (A5) of low frequencies ranging from 0 to 3.90 Hz. Accuracy rates for detailed coefficients were 98.57 and 98.39% for SVM and KNN, respectively; and for detailed coefficients (D5) deriving from the sub-band range (3.90-7.81 Hz). Accuracy rates for MLP and NB classifiers were comparable at 97.11-89.63% and 91.60-81.07% for A5 and D5 coefficients, respectively. In addition, the proposed approach was also applied on public dataset for classification of two cognitive tasks and achieved comparable classification results, i.e., 93.33% accuracy with KNN. The proposed scheme yielded significantly higher classification performances using machine learning classifiers compared to extant quantitative feature extraction. These results suggest the proposed feature extraction method reliably classifies EEG signals recorded during cognitive tasks with a higher degree of accuracy.

Single Trial EEG Patterns for the Prediction of Individual Differences in Fluid Intelligence

Assessing a person's intelligence level is required in many situations, such as career counseling and clinical applications. EEG evoked potentials in oddball task and fluid intelligence score are correlated because both reflect the cognitive processing and attention. A system for prediction of an individual's fluid intelligence level using single trial Electroencephalography (EEG) signals has been proposed. For this purpose, we employed 2D and 3D contents and 34 subjects each for 2D and 3D, which were divided into low-ability (LA) and high-ability (HA) groups using Raven's Advanced Progressive Matrices (RAPM) test. Using visual oddball cognitive task, neural activity of each group was measured and analyzed over three midline electrodes (Fz, Cz, and Pz). To predict whether an individual belongs to LA or HA group, features were extracted using wavelet decomposition of EEG signals recorded in visual oddball task and support vector machine (SVM) was used as a classifier. Two different types of Haar wavelet transform based features have been extracted from the band (0.3 to 30 Hz) of EEG signals. Statistical wavelet features and wavelet coefficient features from the frequency bands 0.0–1.875 Hz (delta low) and 1.875–3.75 Hz (delta high), resulted in the 100 and 98% prediction accuracies, respectively, both for 2D and 3D contents. The analysis of these frequency bands showed clear difference between LA and HA groups. Further, discriminative values of the features have been validated using statistical significance tests and inter-class and intra-class variation analysis. Also, statistical test showed that there was no effect of 2D and 3D content on the assessment of fluid intelligence level. Comparisons with state-of-the-art techniques showed the superiority of the proposed system.

Visual brain activity patterns classification with simultaneous EEG-fMRI: A multimodal approach

BACKGROUND

Classification of the visual information from the brain activity data is a challenging task. Many studies reported in the literature are based on the brain activity patterns using either fMRI or EEG/MEG only. EEG and fMRI considered as two complementary neuroimaging modalities in terms of their temporal and spatial resolution to map the brain activity. For getting a high spatial and temporal resolution of the brain at the same time, simultaneous EEG-fMRI seems to be fruitful.

METHODS

In this article, we propose a new method based on simultaneous EEG-fMRI data and machine learning approach to classify the visual brain activity patterns. We acquired EEG-fMRI data simultaneously on the ten healthy human participants by showing them visual stimuli. Data fusion approach is used to merge EEG and fMRI data. Machine learning classifier is used for the classification purposes.

RESULTS

Results showed that superior classification performance has been achieved with simultaneous EEG-fMRI data as compared to the EEG and fMRI data standalone. This shows that multimodal approach improved the classification accuracy results as compared with other approaches reported in the literature.

CONCLUSIONS

The proposed simultaneous EEG-fMRI approach for classifying the brain activity patterns can be helpful to predict or fully decode the brain activity patterns.

Evaluation of passive polarized stereoscopic 3D display for visual & mental fatigues

Visual and mental fatigues induced by active shutter stereoscopic 3D (S3D) display have been reported using event-related brain potentials (ERP). An important question, that is whether such effects (visual & mental fatigues) can be found in passive polarized S3D display, is answered here. Sixty-eight healthy participants are divided into 2D and S3D groups and subjected to an oddball paradigm after being exposed to S3D videos with passive polarized display or 2D display. The age and fluid intelligence ability of the participants are controlled between the groups. ERP results do not show any significant differences between S3D and 2D groups to find the aftereffects of S3D in terms of visual and mental fatigues. Hence, we conclude that passive polarized S3D display technology may not induce visual and/or mental fatigue which may increase the cognitive load and suppress the ERP components.

P300 correlates with learning & memory abilities and fluid intelligence

Background

Educational psychology research has linked fluid intelligence with learning and memory abilities and neuroimaging studies have specifically associated fluid intelligence with event related potentials (ERPs). The objective of this study is to find the relationship of ERPs with learning and memory recall and predict the memory recall score using P300 (P3) component.

Method

A sample of thirty-four healthy subjects between twenty and thirty years of age was selected to perform three tasks: (1) Raven’s Advanced Progressive Matrices (RAPM) test to assess fluid intelligence; (2) learning and memory task to assess learning ability and memory recall; and (3) the visual oddball task to assess brain-evoked potentials. These subjects were divided into High Ability (HA) and Low Ability (LA) groups based on their RAPM scores. A multiple regression analysis was used to predict the learning & memory recall and fluid intelligence using P3 amplitude and latency.

Results

Behavioral results demonstrated that the HA group learned and recalled 10.89 % more information than did the LA group. ERP results clearly showed that the P3 amplitude of the HA group was relatively larger than that observed in the LA group for both the central and parietal regions of the cerebrum; particularly during the 300–400 ms time window. In addition, a shorter latency for the P3 component was observed at Pz site for the HA group compared to the LA group. These findings agree with previous educational psychology and neuroimaging studies which reported an association between ERPs and fluid intelligence as well as learning performance.

Conclusion

These results also suggest that the P3 component is associated with individual differences in learning and memory recall and further indicate that P3 amplitude might be used as a supporting factor in standard psychometric tests to assess an individual’s learning & memory recall ability; particularly in educational institutions to aid in the predictability of academic skills.

EEG based evaluation of stereoscopic 3D displays for viewer discomfort

BACKGROUND

Consumer preference is rapidly changing from 2D to 3D movies due to the sensational effects of 3D scenes, like those in Avatar and The Hobbit. Two 3D viewing technologies are available: active shutter glasses and passive polarized glasses. However, there are consistent reports of discomfort while viewing in 3D mode where the discomfort may refer to dizziness, headaches, nausea or simply not being able to see in 3D continuously.

METHODS

In this paper, we propose a theory that 3D technology which projects the two images (required for 3D perception) alternatively, cannot provide true 3D visual experience while the 3D technology projecting the two images simultaneously is closest to the human visual system for depth perception. Then we validate our theory by conducting experiments with 40 subjects and analyzing the EEG results of viewing 3D movie clips with passive polarized glasses while the images are projected simultaneously compared to 2D viewing. In addition, subjective feedback of the subjects was also collected and analyzed.

RESULTS

A higher theta and alpha band absolute power is observed across various areas including the occipital lobe for 3D viewing. We also found that the complexity of the signal, e.g. variations in EEG samples over time, increases in 3D as compared to 2D. Various results conclude that working memory, as well as, attention is increased in 3D viewing because of the processing of more data in 3D as compared to 2D. From subjective feedback analysis, 75% of subjects felt comfortable with 3D passive polarized while 25% preferred 3D active shutter technology.

CONCLUSIONS

We conclude that 3D passive polarized technology provides more comfortable visualization than 3D active shutter technology. Overall, 3D viewing is more attractive than 2D due to stereopsis which may cause of high attention and involvement of working memory manipulations.