Effects of spectral smearing of stimuli on the performance of auditory steady-state response-based brain-computer interface

A method for simulation the effect of the reduced frequency resolution of the ear in patients with sensorineural hearing loss

A method for the simulation of reduced frequency resolution of the ear in patients with sensorineural hearing loss is proposed. The method is based upon the ability to adjust it according to the audiogram of a concrete person by frame-by-frame signal processing in the frequency domain. Simulation of the effect of the reduced frequency resolution of the ear is achieved by processing the components of amplitude spectrum of the original sound signal by the "smearing" function. The "smearing" function is formed from the amplitude-frequency characteristics of the auditory filters, which bandwidth is determined by the audiogram of the deaf person. The proposed method is implemented in the MATLAB. An experimental study of the effect of the reduced frequency resolution of the ear using the speech intelligibility test was conducted. The experiment involved 15 people who listened the records processed by the proposed method with various settings and noise conditions. Experimental data have shown that reduced frequency resolution of the ear leads to the deterioration in speech intelligibility, especially in the presence of background noise. Based on the answers of the participants of the experiment, the confusion tables of sounds were compiled, reflecting the fact of indistinguishability of sounds similar in frequency, that confirms the correctness of the proposed method.

Filter bank temporally local canonical correlation analysis for short time window SSVEPs classification

Canonical correlation analysis (CCA) method and its extended methods have been widely and successfully applied to the frequency recognition in SSVEP-based BCI systems. As a state-of-the-art extended method, filter bank canonical correlation analysis has higher accuracy and information transmission rate (ITR) than CCA. However, in the CCA method, the temporally local structure of samples has not been well considered. In this correspondence, we proposed termed temporally local canonical correlation analysis (TCCA). In this new method, the original covariance matrix was replaced by the temporally local covariance matrix. Furthermore, we proposed an improved frequency identification method of filter bank based on TCCA, named filter bank temporally local canonical correlation analysis (FBTCCA). In the offline environment, we used a leave-one-subject-out validation strategy on datasets of ten testees to optimize the parameters of TCCA and FBTCCA and evaluate the two algorithms. The experimental results affirm that TCCA markedly outperformed CCA, and FBTCCA obtained the highest accuracy among the four methods. This study corroborates that TCCA-based approaches have great potential for implementing short time window SSVEP-based BCI systems.

Novel hybrid brain-computer interface system based on motor imagery and P300

Motor imagery (MI) is a mental representation of motor behavior and has been widely used in electroencephalogram based brain-computer interfaces (BCIs). Several studies have demonstrated the efficacy of MI-based BCI-feedback training in post-stroke rehabilitation. However, in the earliest stage of the training, calibration data typically contain insufficient discriminability, resulting in unreliable feedback, which may decrease subjects' motivation and even hinder their training. To improve the performance in the early stages of MI training, a novel hybrid BCI paradigm based on MI and P300 is proposed in this study. In this paradigm, subjects are instructed to imagine writing the Chinese character following the flash order of the desired Chinese character displayed on the screen. The event-related desynchronization/synchronization (ERD/ERS) phenomenon is produced with writing based on one's imagination. Simultaneously, the P300 potential is evoked by the flash of each stroke. Moreover, a fusion method of P300 and MI classification is proposed, in which unreliable P300 classifications are corrected by reliable MI classifications. Twelve healthy naïve MI subjects participated in this study. Results demonstrated that the proposed hybrid BCI paradigm yielded significantly better performance than the single-modality BCI paradigm. The recognition accuracy of the fusion method is significantly higher than that of P300 (p < 0.05) and MI (p < 0.01). Moreover, the training data size can be reduced through fusion of these two modalities.

Mental fatigue level detection based on event related and visual evoked potentials features fusion in virtual indoor environment

The purpose of this work is to set up a model that can estimate the mental fatigue of users based on the fusion of relevant features extracted from Positive 300 (P300) and steady state visual evoked potentials (SSVEP) measured by electroencephalogram. To this end, an experimental protocol describes the induction of P300, SSVEP and mental workload (which leads to mental fatigue by varying time-on-task) in different scenarios where environmental artifacts are controlled (obstacles number, obstacles velocities, ambient luminosity). Ten subjects took part in the experiment (with two suffering from cerebral palsy). Their mission is to navigate along a corridor from a starting point A to a goal point B where specific flickering stimuli are introduced to perform the P300 task. On the other hand, SSVEP task is elicited thanks to 10 Hz flickering lights. Correlated features are considered as inputs to fusion block which estimates mental workload. In order to deal with uncertainties and heterogeneity of P300 and SSVEP features, Dempster-Shafer (D-S) evidential reasoning is introduced. As the goal is to assess the reliability for the estimation of mental fatigue levels, D-S is compared to multi layer perception and linear discriminant analysis. The results show that D-S globally outperforms the other classifiers (although its performance significantly decreases between healthy and palsied groups). Finally we discuss the feasibility of such a fusion proposal in real life situation.

Effects of degree and symmetricity of bilateral spectral smearing, carrier frequency, and subject sex on amplitude of evoked auditory steady-state response signal

The characteristics of an auditory steady-state response (ASSR) signal can be affected by the pathophysiological statuses of the left and right ears, such as a smeared sensation by native spectral smearing owing to sensorineural hearing impairment, because they can affect the perception of the stimulus, the degree of concentration on the stimulus and comfort in concentration. However, to date, few studies have examined the effects of such smeared sensations on the amplitude of the evoked ASSR signal. In this study, we synthesized various auditory stimuli with different degrees of spectral smearing using a hearing loss simulator to match the age of participant groups with different degrees of spectral smearing. We then performed three subjective tests, representing symmetric and asymmetric bilateral spectral smearing, with 16 normal-hearing individuals to observe the effects of the severity and symmetricity of bilateral spectral smearing, the value of the carrier frequency of auditory stimuli, and the sex of the individual on the amplitude in evoked ASSR signals. The experimental results demonstrated the following: (1) the application of spectral smearing to normal sounds may result in amplitude-reduced ASSR signals, (2) the effect of spectral smearing on the amplitude of the ASSR signals is most significant when the degrees of bilateral spectral smearing are asymmetric, (3) the selection of carrier frequency in an auditory stimulus can affect the amplitude of evoked ASSR signals regardless of the degree of spectral smearing, and (4) the sex of the individual can affect the amplitude of the evoked ASSR signal in various test conditions. The results of this study can help estimate the effects of smeared sensation by spectral smearing owing to sensorineural hearing impairment on the amplitude of evoked ASSR signals.

Low and high gamma auditory steady-states in response to 440 Hz carrier chirp-modulated tones show no signs of attentional modulation

Auditory steady-state responses (ASSRs) are increasingly used as a neurodiagnostical tool and in neurotechnological applications where it is important to test responses at different frequencies. We aimed to evaluate EEG responses to a low-frequency carrier (440 Hz) tone modulated with 500 ms chirps at 1-120 Hz and to test the ability of stimulation to highlight the low and high gamma band activity. Increasing and decreasing modulation rates were applied. Subjective pleasantness of chirps and attentional effects on ASSRs to chirps were assessed. Our results suggest that brief low-frequency tones modulated with chirps can be used to evoke ASSRs and to test responses at low and high gamma frequencies. Moreover, chirps are perceived as moderately arousing and neutrally pleasant, while ASSRs to these sounds are not sensitive to attentional modulation. The abovementioned findings make chirp stimulation suitable for use in populations with increased perceptual sensitivity to auditory stimuli, for instance like patients with schizophrenia.