Convolutional Correlation Analysis for Enhancing the Performance of SSVEP-Based Brain-Computer Interface

Currently, most of the high-performance models for frequency recognition of steady-state visual evoked potentials (SSVEPs) are linear. However, SSVEPs collected from different channels can have non-linear relationship among each other. Linearly combining electroencephalogram (EEG) from multiple channels is not the most accurate solution in SSVEPs classification. To further improve the performance of SSVEP-based brain-computer interface (BCI), we propose a convolutional neural network-based non-linear model, i.e. convolutional correlation analysis (Conv-CA). Different from pure deep learning models, Conv-CA use convolutional neural networks (CNNs) at the top of a self-defined correlation layer. The CNNs function on how to transform multiple channel EEGs into a single EEG signal. The correlation layer calculates the correlation coefficients between the transformed single EEG signal and reference signals. The CNNs provide non-linear operations to combine EEGs in different channels and different time. And the correlation layer constrains the fitting space of the deep learning model. A comparison study between the proposed Conv-CA method and the task-related component analysis (TRCA) based methods is conducted. Both methods are validated on a 40-class SSVEP benchmark dataset recorded from 35 subjects. The study verifies that the Conv-CA method significantly outperforms the TRCA-based methods. Moreover, Conv-CA has good explainability since its inputs of the correlation layer can be analyzed for visualizing what the model learnt from the data. Conv-CA is a non-linear extension of spatial filters. Its CNN structures can be further explored and tuned for reaching a better performance. The structure of combining neural networks and unsupervised features has the potential to be applied to the classification of other signals.

Evaluation of a BSS algorithm for artifacts rejection in epileptic seizure detection

A data efficient blind sources separation (BSS) algorithm has been applied to preprocess intracranial EEG (ECoG) for artifact rejection. After artifacts correction a recurrence time statistics T1 feature was evaluated from the 'cleaned' data. Seizure detection performance was compared between BSS preprocessing and without preprocessing. Test results show that in a data set, for a detection rate of 96%, the false alarm rate dropped from 0.13 per hour without BSS preprocessing to 0.08 with preprocessing. For the other set of data, the false alarm rate dropped from 0.34 to 0.21 at a detection rate of 100%.

Comparison between human and artificial neural network detection of Laplacian-derived electroencephalographic activity related to unilateral voluntary movements

A back-propagation artificial neural network (ANN) was tested to verify its capacity to select different classes of single trials (STs) based on the spatial information content of electroencephalographic activity related to voluntary unilateral finger movements. The rationale was that ipsilateral and contralateral primary sensorimotor cortex can be involved in a nonstationary way in the control of unilateral voluntary movements. The movement-related potentials were surface Laplacian-transformed (SL) to reduce head volume conductor effects and to model the response of the primary sensorimotor cortex. The ANN sampled the SL from four or two central channels overlying the primary motor area of both sides in the period of 80 ms preceding the electromyographic response onset in the active muscle. The performance of the ANN was evaluated statistically by calculating the percentage value of agreement between the STs classified by the ANN and those of two investigators (used as a reference). The results showed that both investigator and ANN were capable of selecting STs with the SL maximum in the central area contralateral to the movement (contralateral STs, about 25%), STs with considerable SL values also in the ipsilateral central area (bilateral STs, about 50%), and STs with neither the contralateral nor bilateral pattern ("spatially incoherent" single trials; about 25%). The maximum agreement (64-84%) between the ANN and the investigator was obtained when the ANN used four spatial inputs (P < 0.0000001). Importantly, the common means of all single trials showed a weak or absent ipsilateral response. These results may suggest that a back-propagation ANN could select EEG single trials showing stationary and nonstationary responses of the primary sensorimotor cortex, based on the same spatial criteria as the experimenter.

Monitoring working memory load during computer-based tasks with EEG pattern recognition methods

We assessed working memory load during computer use with neural network pattern recognition applied to EEG spectral features. Eight participants performed high-, moderate-, and low-load working memory tasks. Frontal theta EEG activity increased and alpha activity decreased with increasing load. These changes probably reflect task difficulty-related increases in mental effort and the proportion of cortical resources allocated to task performance. In network analyses, test data segments from high and low load levels were discriminated with better than 95% accuracy. More than 80% of test data segments associated with a moderate load could be discriminated from high- or low-load data segments. Statistically significant classification was also achieved when applying networks trained with data from one day to data from another day, when applying networks trained with data from one task to data from another task, and when applying networks trained with data from a group of participants to data from new participants. These results support the feasibility of using EEG-based methods for monitoring cognitive load during human-computer interaction.

Review of neural network applications in sleep research

To find a better automated sleep-wake staging system for human analyses of numerous polygraphic records is an interesting challenge in sleep research. Over the last few decades, many automated systems have been developed but none are universally applicable. Improvements in computer technology coupled with artificial neural networks based systems (connectionist models) are responsible for new data processing approaches. Despite extensive use of connectionist models in biological data processing, in the past, the field of sleep research appeared to have neglected this approach. Only a few sleep-wake staging systems based on neural network technology have been developed. This paper reviews the current use of artificial neural networks in sleep research. Following a brief presentation of neural network technology, each of the existing system is described and attention drawn to the heterogeneity of the different processing approaches in sleep research. The high performances observed with systems based on neural networks highlight the need to integrate these tools into the field of sleep research.

Towards measurement of brain function in operational environments

In operational environments that demand sustained vigilance or that involve multiple tasks competing for limited attentional resources, continuous monitoring of the mental state of the operator could decrease the potential for serious errors and provide valuable information concerning the ergonomics of the tasks being performed. There is widespread discussion and appreciation of the basic feasibility of utilizing neurophysiological measurements to derive accurate, reliable, rapid and unobtrusive assessments of mental state. However, progress in transitioning this idea into practical applications has been impeded by the fact that at present no convenient, inexpensive and effective means exists to derive a meaningful index of brain activity outside of laboratory settings. In this paper, we review some recent advances in recording technology and signal processing methods that will help overcome this limitation. For example, rapid progress is being made in the engineering of recording systems that are small, rugged, portable and easy-to-use, and thus suitable for deployment in operational environments. Progress is also being made in the development of signal processing algorithms for detecting and correcting recording artifacts and for increasing the amount of useful information that can be derived from brain signals. Finally, results from basic research studies suggest that accurate and reliable inferences about the mental load and alertness of an individual can be derived from neurophysiological measures in a practical fashion. These research and engineering successes suggest that it is reasonable to expect that in the near term a basic enabling technology will be deployed that will permit routine measurement of brain function in operational environments.

Neural network assisted cardiac auscultation

Traditional cardiac auscultation involves a great deal of interpretive skill. Neural networks were trained as phonocardiographic classifiers to determine their viability in this rôle. All networks had three layers and were trained by backpropagation using only the heart sound amplitude envelope as input. The main aspect of the study was to determine what topologies, gain and momentum factors lead to efficient training for this application. Neural networks which are trained with heart sound classes of greater similarity were found to be less likely to converge to a solution. A prototype normal/abnormal classifier was also developed which provided excellent classification accuracy despite the sparse nature of the training data. Future directions for the development of a full-scale computer-assisted phonocardiographic classifier are also considered.

Neural network approach in multichannel auditory event-related potential analysis

Even though there are presently no clearly defined criteria for the assessment of P300 event-related potential (ERP) abnormality, it is strongly indicated through statistical analysis that such criteria exist for classifying control subjects and patients with diseases resulting in neuropsychological impairment such as multiple sclerosis (MS). We have demonstrated the feasibility of artificial neural network (ANN) methods in classifying ERP waveforms measured at a single channel (Cz) from control subjects and MS patients. In this paper, we report the results of multichannel ERP analysis and a modified network analysis methodology to enhance automation of the classification rule extraction process. The proposed methodology significantly reduces the work of statistical analysis. It also helps to standardize the criteria of P300 ERP assessment and facilitate the computer-aided analysis on neuropsychological functions.

Neural network analysis of the P300 event-related potential in multiple sclerosis

Neural network analysis is sensitive to subtle changes in patterns of data. We hypothesized that a disease process which can cause impairment of cortical function such as multiple sclerosis (MS) would affect the P300 cognitive evoked potential (P300) in a manner detectable by a feedforward backpropagation neural network. Such a network was trained using a learning data set consisting of 101 P300 wave forms (from 26 MS patients and 26 normal controls). The network was then used to classify a randomly selected test data set of 20 studies (2 studies each of 5 MS patients and 5 controls) to which it had not been previously exposed, with an average accuracy (MS = abnormal, control = normal) of 81% for a single midline electrode, increasing to 90% using 3 midline electrodes in a jury system. Neural network analysis can be of help in distinguishing normal (control) P300 from abnormal (MS) P300.

Neural network based classification of single-trial EEG data

Standard Back Propagation (BP), Partially Recurrent (PR) and Cascade-Correlation (CC) neural networks were used to predict the side of finger movement on the basis of non-averaged single trial multi-channel EEG data recorded prior to movement. From these EEG data, power values were calculated and used as parameters for classification. The results obtained on three subjects show that the Cascade-Correlation neural network is an appropriate choice for neural network based classification of spatio-temporal single-trial EEG patterns. It is fast, stable and able to discover and recognize underlying dynamics of rhythmic activities within the alpha band which precede execution of hand movements.

Identification of peak V in brainstem auditory evoked potentials with neural networks

A feature extractor for determining the latency of peak V in brainstem auditory evoked potentials (BAEPs) is presented in this paper. A feature extractor that combines artificial neural networks with an algorithmic approach is presented. It consists of a series of small neural networks that have to make simple decisions. Each neural network decides what part of the input pattern contains the peak, and the algorithm passes that part of the pattern to the next neural network; in this way the size of the input patterns decreases during the process, and the last neural network determines the exact location of the peak. An optimal configuration of neural networks could determine the latencies of peak V in all synthetic evoked potentials correctly. With real evoked potentials, the networks yield results that comply with the opinion of a human expert in 80 +/- 6% of the cases.

Seeing through the skull: advanced EEGs use MRIs to accurately measure cortical activity from the scalp

There is a vast amount of untapped spatial information in scalp-recorded EEGs. Measuring this information requires use of many electrodes and application of spatial signal enhancing procedures to reduce blur distortion due to transmission through the skull and other tissues. Recordings with 124 electrodes are now routinely made, and spatial signal enhancing techniques have been developed. The most advanced of these techniques uses information from a subject's MRI to correct blur distortion, in effect providing a measure of the actual cortical potential distribution. Examples of these procedures are presented, including a validation from subdural recordings in an epileptic patient. Examples of equivalent dipole modeling of the somatosensory evoked potential are also presented in which two adjacent fingers are clearly separated. These results demonstrate that EEGs can provide images of superficial cortical electrical activity with spatial detail approaching that of O15 PET scans. Additionally, equivalent dipole modeling with EEGs appears to have the same degree of spatial resolution as that reported for MEGs. Considering that EEG technology costs ten to fifty times less than other brain imaging modalities, that it is completely harmless, and that recordings can be made in naturalistic settings for extended periods of time, a greater investment in advancing EEG technology seems very desirable.

Effects of prolonged mental work on functional brain topography

Topographic patterns of event-related covariance between electrodes were measured from subjects performing a difficult memory and fine-motor control task for 10-14 h. Striking changes occurred in the patterns after subjects performed the task for an average of 7-9 h, but before performance deteriorated. Pattern strength was reduced in a fraction-of-a-second-long response preparation interval over midline precentral areas and over the entire left hemisphere. By contrast, pattern strength in a succeeding response inhibition interval was reduced over all areas. The pattern changed least in an intervening interval associated with visual-stimulus processing. This suggests that, in addition to the well-known global reduction in neuroelectric signal strength, functional neural networks are selectively affected by sustained mental work in specific fraction-of-a-second task intervals.

Beyond topographic mapping: towards functional-anatomical imaging with 124-channel EEGs and 3-D MRIs

A functional-anatomical brain scanner that has a temporal resolution of less than a hundred milliseconds is needed to measure the neural substrate of higher cognitive functions in healthy people and neurological and psychiatric patients. Electrophysiological techniques have the requisite temporal resolution but their potential spatial resolution has been not realized. Here we briefly review progress in increasing the spatial detail of scalp-recorded EEGs and in registering this functional information with anatomical models of a person's brain. We describe methods and systems for 124-channel EEGs and magnetic resonance image (MRI) modeling, and present first results of the integration of equivalent-dipole EEG models of somatosensory stimulation with 3-D MRI brain models.

Event-related covariances during a bimanual visuomotor task. II. Preparation and feedback

Event-related covariance (ERC) patterns were computed from pre-stimulus and feedback intervals of a bimanual, visuomotor judgment task performed by 7 right-handed men. Late contingent negative variation (CNV) ERC patterns that preceded subsequently accurate right- or left-hand responses differed from patterns that preceded subsequently inaccurate responses. Recordings from electrodes placed at left frontal, midline antero-central, and appropriately contralateral central and parietal sites were prominent in ERC patterns of subsequently accurate performances. This suggests that a distributed cortical 'preparatory network,' composed of distinct cognitive, integrative motor, somesthetic, and motor components, is essential for accurate visuomotor performance. ERC patterns related to feedback about accurate and inaccurate responses were similar to each other in the interval immediately after feedback onset, but began to differ in an interval spanning an early P300 peak. The difference became even greater in an interval spanning a late P300 peak. For both early and late P300 peaks, ERC patterns following feedback about inaccurate performance involved more frontal sites than did those following feedback about accurate performance. Together with the stimulus- and response-locked results presented in part I, results of this study on the preparatory and feedback periods suggest that ERCs show salient features of the rapidly shifting, functional cortical networks that are responsible for simple cognitive tasks. ERCs thus provide a new perspective on information processing in the human brain in relation to behavior--a perspective that supplements conventional EEG and ERP procedures.

Dynamic functional topography of cognitive tasks

Improved neuroelectric recording and analysis tools are yielding increasingly specific information about the spatial and temporal features of neurocognitive processes. Such tools include recordings with up to 125 channels, digital signal processing techniques, and correlation of neuroelectric measures with anatomical information from magnetic resonance images. These tools, and their application to the study of cognitive functions, are presented in this paper.