Multimodal fusion of EMG and vision for human grasp intent inference in prosthetic hand control

Objective: For transradial amputees, robotic prosthetic hands promise to regain the capability to perform daily living activities. Current control methods based on physiological signals such as electromyography (EMG) are prone to yielding poor inference outcomes due to motion artifacts, muscle fatigue, and many more. Vision sensors are a major source of information about the environment state and can play a vital role in inferring feasible and intended gestures. However, visual evidence is also susceptible to its own artifacts, most often due to object occlusion, lighting changes, etc. Multimodal evidence fusion using physiological and vision sensor measurements is a natural approach due to the complementary strengths of these modalities. Methods: In this paper, we present a Bayesian evidence fusion framework for grasp intent inference using eye-view video, eye-gaze, and EMG from the forearm processed by neural network models. We analyze individual and fused performance as a function of time as the hand approaches the object to grasp it. For this purpose, we have also developed novel data processing and augmentation techniques to train neural network components. Results: Our results indicate that, on average, fusion improves the instantaneous upcoming grasp type classification accuracy while in the reaching phase by 13.66% and 14.8%, relative to EMG (81.64% non-fused) and visual evidence (80.5% non-fused) individually, resulting in an overall fusion accuracy of 95.3%. Conclusion: Our experimental data analyses demonstrate that EMG and visual evidence show complementary strengths, and as a consequence, fusion of multimodal evidence can outperform each individual evidence modality at any given time.

Functional connectivity alterations in traumatic brain injury patients with late seizures

PTE is a neurological disorder characterized by recurrent and spontaneous epileptic seizures. PTE is a major public health problem occurring in 2-50% of TBI patients. Identifying PTE biomarkers is crucial for the development of effective treatments. Functional neuroimaging studies in patients with epilepsy and in epileptic rodents have observed that abnormal functional brain activity plays a role in the development of epilepsy. Network representations of complex systems ease quantitative analysis of heterogeneous interactions within a unified mathematical framework. In this work, graph theory was used to study resting state functional magnetic resonance imaging (rs-fMRI) and reveal functional connectivity abnormalities that are associated with seizure development in traumatic brain injury (TBI) patients. We examined rs-fMRI of 75 TBI patients from Epilepsy Bioinformatics Study for Antiepileptogenic Therapy (EpiBioS4Rx) which aims to identify validated Post-traumatic epilepsy (PTE) biomarkers and antiepileptogenic therapies using multimodal and longitudinal data acquired from 14 international sites. The dataset includes 28 subjects who had at least one late seizure after TBI and 47 subjects who had no seizures within 2 years post-injury. Each subject's neural functional network was investigated by computing the correlation between the low frequency time series of 116 regions of interest (ROIs). Each subject's functional organization was represented as a network consisting of nodes, brain regions, and edges that show the relationship between the nodes. Then, several graph measures concerning the integration and the segregation of the functional brain networks were extracted in order to highlight changes in functional connectivity between the two TBI groups. Results showed that the late seizure-affected group had a compromised balance between integration and segregation and presents functional networks that are hyperconnected, hyperintegrated but at the same time hyposegregated compared with seizure-free patients. Moreover, TBI subjects who developed late seizures had more low betweenness hubs.

Similarity of hand muscle synergies elicited by transcranial magnetic stimulation and those found during voluntary movement

Converging evidence in human and animal models suggests that exogenous stimulation of the motor cortex (M1) elicits responses in the hand with similar modular structure to that found during voluntary grasping movements. The aim of this study was to establish the extent to which modularity in muscle responses to transcranial magnetic stimulation (TMS) to M1 resembles modularity in muscle activation during voluntary hand movements involving finger fractionation. Electromyography (EMG) was recorded from eight hand-forearm muscles in eight healthy individuals. Modularity was defined using non-negative matrix factorization to identify low-rank approximations (spatial muscle synergies) of the complex activation patterns of EMG data recorded during high-density TMS mapping of M1 and voluntary formation of gestures in the American Sign Language alphabet. Analysis of synergies revealed greater than chance similarity between those derived from TMS and those derived from voluntary movement. Both data sets included synergies dominated by single intrinsic hand muscles presumably to meet the demand for highly fractionated finger movement. These results suggest that corticospinal connectivity to individual intrinsic hand muscles may be combined with modular multimuscle activation via synergies in the formation of hand postures.NEW & NOTEWORTHY This is the first work to examine the similarity of modularity in hand muscle responses to transcranial magnetic stimulation (TMS) of the motor cortex and that derived from voluntary hand movement. We show that TMS-elicited muscle synergies of the hand, measured at rest, reflect those found in voluntary behavior involving finger fractionation. This work provides a basis for future work using TMS to investigate muscle activation modularity in the human motor system.

Inference of Upcoming Human Grasp Using EMG During Reach-to-Grasp Movement

Electromyography (EMG) data has been extensively adopted as an intuitive interface for instructing human-robot collaboration. A major challenge to the real-time detection of human grasp intent is the identification of dynamic EMG from hand movements. Previous studies predominantly implemented the steady-state EMG classification with a small number of grasp patterns in dynamic situations, which are insufficient to generate differentiated control regarding the variation of muscular activity in practice. In order to better detect dynamic movements, more EMG variability could be integrated into the model. However, only limited research was conducted on such detection of dynamic grasp motions, and most existing assessments on non-static EMG classification either require supervised ground-truth timestamps of the movement status or only contain limited kinematic variations. In this study, we propose a framework for classifying dynamic EMG signals into gestures and examine the impact of different movement phases, using an unsupervised method to segment and label the action transitions. We collected and utilized data from large gesture vocabularies with multiple dynamic actions to encode the transitions from one grasp intent to another based on natural sequences of human grasp movements. The classifier for identifying the gesture label was constructed afterward based on the dynamic EMG signal, with no supervised annotation of kinematic movements required. Finally, we evaluated the performances of several training strategies using EMG data from different movement phases and explored the information revealed from each phase. All experiments were evaluated in a real-time style with the performance transitions presented over time.

Improved Training Efficiency for Retinopathy of Prematurity Deep Learning Models Using Comparison versus Class Labels

Purpose

To compare the efficacy and efficiency of training neural networks for medical image classification using comparison labels indicating relative disease severity versus diagnostic class labels from a retinopathy of prematurity (ROP) image dataset.

Design

Evaluation of diagnostic test or technology.

Participants

Deep learning neural networks trained on expert-labeled wide-angle retinal images obtained from patients undergoing diagnostic ROP examinations obtained as part of the Imaging and Informatics in ROP (i-ROP) cohort study.

Methods

Neural networks were trained with either class or comparison labels indicating plus disease severity in ROP retinal fundus images from 2 datasets. After training and validation, all networks underwent evaluation using a separate test dataset in 1 of 2 binary classification tasks: normal versus abnormal or plus versus nonplus.

Main Outcome Measures

Area under the receiver operating characteristic curve (AUC) values were measured to assess network performance.

Results

Given the same number of labels, neural networks learned more efficiently by comparison, generating significantly higher AUCs in both classification tasks across both datasets. Similarly, given the same number of images, comparison learning developed networks with significantly higher AUCs across both classification tasks in 1 of 2 datasets. The difference in efficiency and accuracy between models trained on either label type decreased as the size of the training set increased.

Conclusions

Comparison labels individually are more informative and more abundant per sample than class labels. These findings indicate a potential means of overcoming the common obstacle of data variability and scarcity when training neural networks for medical image classification tasks.

Feedback Related Potentials for EEG-Based Typing Systems

Error related potentials (ErrP), which are elicited in the EEG in response to a perceived error, have been used for error correction and adaption in the event related potential (ERP)-based brain computer interfaces designed for typing. In these typing interfaces, ERP evidence is collected in response to a sequence of stimuli presented usually in the visual form and the intended user stimulus is probabilistically inferred (stimulus with highest probability) and presented to the user as the decision. If the inferred stimulus is incorrect, ErrP is expected to be elicited in the EEG. Early approaches to use ErrP in the design of typing interfaces attempt to make hard decisions on the perceived error such that the perceived error is corrected and either the sequence of stimuli are repeated to obtain further ERP evidence, or without further repetition the stimulus with the second highest probability is presented to the user as the decision of the system. Moreover, none of the existing approaches use a language model to increase the performance of typing. In this work, unlike the existing approaches, we study the potential benefits of fusing feedback related potentials (FRP), a form of ErrP, with ERP and context information (language model, LM) in a Bayesian fashion to detect the user intent. We present experimental results based on data from 12 healthy participants using RSVP Keyboard™ to complete a copy-phrase-task. Three paradigms are compared: [P1] uses only ERP/LM Bayesian fusion; [P2] each RSVP sequence is appended with the top candidate in the alphabet according to posterior after ERP evidence fusion; corresponding FRP is then incorporated; and [P3] the top candidate is shown as a prospect to generate FRP evidence only if its posterior exceeds a threshold. Analyses indicate that ERP/LM/FRP evidence fusion during decision making yields significant speed-accuracy benefits for the user.

Lesion Normalization and Supervised Learning in Post-traumatic Seizure Classification with Diffusion MRI

Traumatic brain injury (TBI) is a serious condition, potentially causing seizures and other lifelong disabilities. Patients who experience at least one seizure one week after TBI (late seizure) are at high risk for lifelong complications of TBI, such as post-traumatic epilepsy (PTE). Identifying which TBI patients are at risk of developing seizures remains a challenge. Although magnetic resonance imaging (MRI) methods that probe structural and functional alterations after TBI are promising for biomarker detection, physical deformations following moderate-severe TBI present problems for standard processing of neuroimaging data, complicating the search for biomarkers. In this work, we consider a prediction task to identify which TBI patients will develop late seizures, using fractional anisotropy (FA) features from white matter tracts in diffusion-weighted MRI (dMRI). To understand how best to account for brain lesions and deformations, four preprocessing strategies are applied to dMRI, including the novel application of a lesion normalization technique to dMRI. The pipeline involving the lesion normalization technique provides the best prediction performance, with a mean accuracy of 0.819 and a mean area under the curve of 0.785. Finally, following statistical analyses of selected features, we recommend the dMRI alterations of a certain white matter tract as a potential biomarker.

Stochastic Mutual Information Gradient Estimation for Dimensionality Reduction Networks

Feature ranking and selection is a widely used approach in various applications of supervised dimensionality reduction in discriminative machine learning. Nevertheless there exists significant evidence on feature ranking and selection algorithms based on any criterion leading to potentially sub-optimal solutions for class separability. In that regard, we introduce emerging information theoretic feature transformation protocols as an end-to-end neural network training approach. We present a dimensionality reduction network (MMINet) training procedure based on the stochastic estimate of the mutual information gradient. The network projects high-dimensional features onto an output feature space where lower dimensional representations of features carry maximum mutual information with their associated class labels. Furthermore, we formulate the training objective to be estimated non-parametrically with no distributional assumptions. We experimentally evaluate our method with applications to high-dimensional biological data sets, and relate it to conventional feature selection algorithms to form a special case of our approach.

Geometric Analysis of Uncertainty Sampling for Dense Neural Network Layer

For model adaptation of fully connected neural network layers, we provide an information geometric and sample behavioral active learning uncertainty sampling objective analysis. We identify conditions under which several uncertainty-based methods have the same performance and show that such conditions are more likely to appear in the early stages of learning. We define riskier samples for adaptation, and demonstrate that, as the set of labeled samples increases, margin-based sampling outperforms other uncertainty sampling methods by preferentially selecting these risky samples. We support our derivations and illustrations with experiments using Meta-Dataset, a benchmark for few-shot learning. We compare uncertainty-based active learning objectives using features produced by SimpleCNAPS (a state-of-the-art few-shot classifier) as input for a fully-connected adaptation layer. Our results indicate that margin-based uncertainty sampling achieves similar performance as other uncertainty based sampling methods with fewer labelled samples as discussed in the novel geometric analysis.

EEG-based texture roughness classification in active tactile exploration with invariant representation learning networks

During daily activities, humans use their hands to grasp surrounding objects and perceive sensory information which are also employed for perceptual and motor goals. Multiple cortical brain regions are known to be responsible for sensory recognition, perception and motor execution during sensorimotor processing. While various research studies particularly focus on the domain of human sensorimotor control, the relation and processing between motor execution and sensory processing is not yet fully understood. Main goal of our work is to discriminate textured surfaces varying in their roughness levels during active tactile exploration using simultaneously recorded electroencephalogram (EEG) data, while minimizing the variance of distinct motor exploration movement patterns. We perform an experimental study with eight healthy participants who were instructed to use the tip of their dominant hand index finger while rubbing or tapping three different textured surfaces with varying levels of roughness. We use an adversarial invariant representation learning neural network architecture that performs EEG-based classification of different textured surfaces, while simultaneously minimizing the discriminability of motor movement conditions (i.e., rub or tap). Results show that the proposed approach can discriminate between three different textured surfaces with accuracies up to 70%, while suppressing movement related variability from learned representations.

Disentangled Adversarial Autoencoder for Subject-Invariant Physiological Feature Extraction

Recent developments in biosignal processing have enabled users to exploit their physiological status for manipulating devices in a reliable and safe manner. One major challenge of physiological sensing lies in the variability of biosignals across different users and tasks. To address this issue, we propose an adversarial feature extractor for transfer learning to exploit disentangled universal representations. We consider the trade-off between task-relevant features and user-discriminative information by introducing additional adversary and nuisance networks in order to manipulate the latent representations such that the learned feature extractor is applicable to unknown users and various tasks. Results on cross-subject transfer evaluations exhibit the benefits of the proposed framework, with up to 8.8% improvement in average accuracy of classification, and demonstrate adaptability to a broader range of subjects.

Motor Cortex Mapping using Active Gaussian Processes

One important application of transcranial magnetic stimulation (TMS) is to map cortical motor topography by spatially sampling the motor cortex, and recording motor evoked potentials (MEP) with surface electromyography. Standard approaches to TMS mapping involve repetitive stimulations at different loci spaced on a (typically 1 cm) grid on the scalp. These mappings strategies are time consuming and responsive sites are typically sparse. Furthermore, the long time scale prevents measurement of transient cortical changes, and is poorly tolerated in clinical populations. An alternative approach involves using the TMS mapper expertise to exploit the map's sparsity through the use of feedback of MEPs to decide which loci to stimulate. In this investigation, we propose a novel active learning method to automatically infer optimal future stimulus loci in place of user expertise. Specifically, we propose an active Gaussian Process (GP) strategy with loci selection criteria such as entropy and mutual information (MI). The proposed method twists the usual entropy- and MI-based selection criteria by modeling the estimated MEP field, i.e., the GP mean, as a Gaussian random variable itself. By doing so, we include MEP amplitudes in the loci selection criteria which would be otherwise completely independent of the MEP values. Experimental results using real data shows that the proposed strategy can greatly outperform competing methods when the MEP variations are mostly conned in a sub-region of the space.

Mapping Motor Cortex Stimulation to Muscle Responses: A Deep Neural Network Modeling Approach

A deep neural network (DNN) that can reliably model muscle responses from corresponding brain stimulation has the potential to increase knowledge of coordinated motor control for numerous basic science and applied use cases. Such cases include the understanding of abnormal movement patterns due to neurological injury from stroke, and stimulation based interventions for neurological recovery such as paired associative stimulation. In this work, potential DNN models are explored and the one with the minimum squared errors is recommended for the optimal performance of the M2M-Net, a network that maps transcranial magnetic stimulation of the motor cortex to corresponding muscle responses, using: a finite element simulation, an empirical neural response profile, a convolutional autoencoder, a separate deep network mapper, and recordings of multi-muscle activation. We discuss the rationale behind the different modeling approaches and architectures, and contrast their results. Additionally, to obtain a comparative insight of the trade-o between complexity and performance analysis, we explore different techniques, including the extension of two classical information criteria for M2M-Net. Finally, we find that the model analogous to mapping the motor cortex stimulation to a combination of direct and synergistic connection to the muscles performs the best, when the neural response profile is used at the input.

Siamese neural networks for continuous disease severity evaluation and change detection in medical imaging

Using medical images to evaluate disease severity and change over time is a routine and important task in clinical decision making. Grading systems are often used, but are unreliable as domain experts disagree on disease severity category thresholds. These discrete categories also do not reflect the underlying continuous spectrum of disease severity. To address these issues, we developed a convolutional Siamese neural network approach to evaluate disease severity at single time points and change between longitudinal patient visits on a continuous spectrum. We demonstrate this in two medical imaging domains: retinopathy of prematurity (ROP) in retinal photographs and osteoarthritis in knee radiographs. Our patient cohorts consist of 4861 images from 870 patients in the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) cohort study and 10,012 images from 3021 patients in the Multicenter Osteoarthritis Study (MOST), both of which feature longitudinal imaging data. Multiple expert clinician raters ranked 100 retinal images and 100 knee radiographs from excluded test sets for severity of ROP and osteoarthritis, respectively. The Siamese neural network output for each image in comparison to a pool of normal reference images correlates with disease severity rank (ρ = 0.87 for ROP and ρ = 0.89 for osteoarthritis), both within and between the clinical grading categories. Thus, this output can represent the continuous spectrum of disease severity at any single time point. The difference in these outputs can be used to show change over time. Alternatively, paired images from the same patient at two time points can be directly compared using the Siamese neural network, resulting in an additional continuous measure of change between images. Importantly, our approach does not require manual localization of the pathology of interest and requires only a binary label for training (same versus different). The location of disease and site of change detected by the algorithm can be visualized using an occlusion sensitivity map-based approach. For a longitudinal binary change detection task, our Siamese neural networks achieve test set receiving operator characteristic area under the curves (AUCs) of up to 0.90 in evaluating ROP or knee osteoarthritis change, depending on the change detection strategy. The overall performance on this binary task is similar compared to a conventional convolutional deep-neural network trained for multi-class classification. Our results demonstrate that convolutional Siamese neural networks can be a powerful tool for evaluating the continuous spectrum of disease severity and change in medical imaging.

Classification and comparison via neural networks

We consider learning from comparison labels generated as follows: given two samples in a dataset, a labeler produces a label indicating their relative order. Such comparison labels scale quadratically with the dataset size; most importantly, in practice, they often exhibit lower variance compared to class labels. We propose a new neural network architecture based on siamese networks to incorporate both class and comparison labels in the same training pipeline, using Bradley-Terry and Thurstone loss functions. Our architecture leads to a significant improvement in predicting both class and comparison labels, increasing classification AUC by as much as 35% and comparison AUC by as much as 6% on several real-life datasets. We further show that, by incorporating comparisons, training from few samples becomes possible: a deep neural network of 5.9 million parameters trained on 80 images attains a 0.92 AUC when incorporating comparisons.

An experimental and computational framework for modeling multi-muscle responses to transcranial magnetic stimulation of the human motor cortex

Current knowledge of coordinated motor control of multiple muscles is derived primarily from invasive stimulation-recording techniques in animal models. Similar studies are not generally feasible in humans, so a modeling framework is needed to facilitate knowledge transfer from animal studies. We describe such a framework that uses a deep neural network model to map finite element simulation of transcranial magnetic stimulation induced electric fields (E-fields) in motor cortex to recordings of multi-muscle activation. Critically, we show that model generalization is improved when we incorporate empirically derived physiological models for E-field to neuron firing rate and low-dimensional control via muscle synergies.

Adversarial Deep Learning in EEG Biometrics

Deep learning methods for person identification based on electroencephalographic (EEG) brain activity encounters the problem of exploiting the temporally correlated structures or recording session specific variability within EEG. Furthermore, recent methods have mostly trained and evaluated based on single session EEG data. We address this problem from an invariant representation learning perspective. We propose an adversarial inference approach to extend such deep learning models to learn session-invariant person-discriminative representations that can provide robustness in terms of longitudinal usability. Using adversarial learning within a deep convolutional network, we empirically assess and show improvements with our approach based on longitudinally collected EEG data for person identification from half-second EEG epochs.

Transfer learning using low-dimensional subspaces for EMG-based classification of hand posture

This study proposes a novel approach for evaluating the task invariance of muscle synergies, vital for potential implementation in improving prosthetic hand control. We do this by using a transfer learning paradigm to test for invariance across a relatively small set of hand/forearm muscle synergies, derived from electromyographic (EMG) activation patterns during voluntary behaviors such as finger spelling and grasp mimicking postures and unconstrained exploration. EMG for each task were decomposed using non-negative matrix factorization into synergy and weight matrices, and cross-task weights for each task were then reconstructed by employing the base matrices from different tasks. Support Vector Machine and Extreme Learning Machine classifiers were used to classify the resulting weights in order to compare their performance, as well as their behaviors as a function of synergy rank. Both algorithms showed robust and significantly higher performance, compared to two distinct randomized controls, with lower rank EMG representations, both within and between tasks/postures, supporting hypotheses of functional invariance of multi-muscle synergies. Our results suggest that this invariance could be leveraged to efficiently calibrate postures for prosthetic hand implementation by transferring learned EMG patterns from unconstrained movements to other tasks.

Robust Fusion of c-VEP and Gaze

Brain computer interfaces (BCIs) are one of the developing technologies, serving as a communication interface for people with neuromuscular disorders. Electroencephalography (EEG) and gaze signals are among the commonly used inputs for the user intent classification problem arising in BCIs. Fusing different types of input modalities, i.e. EEG and gaze, is an obvious but effective solution for achieving high performance on this problem. Even though there are some simplistic approaches for fusing these two evidences, a more effective method is required for classification performances and speeds suitable for real-life scenarios. One of the main problems that is left unrecognized is highly noisy real-life data. In the context of the BCI framework utilized in this work, noisy data stem from user error in the form of tracking a nontarget stimuli, which in turn results in misleading EEG and gaze signals. We propose a method for fusing aforementioned evidences in a probabilistic manner that is highly robust against noisy data. We show the performance of the proposed method on real EEG and gaze data for different configurations of noise control variables. Compared to the regular fusion method, robust method achieves up to 15% higher classification accuracy.

Optimal Query Selection Using Multi-Armed Bandits

Query selection for latent variable estimation is conventionally performed by opting for observations with low noise or optimizing information theoretic objectives related to reducing the level of estimated uncertainty based on the current best estimate. In these approaches, typically the system makes a decision by leveraging the current available information about the state. However, trusting the current best estimate results in poor query selection when truth is far from the current estimate, and this negatively impacts the speed and accuracy of the latent variable estimation procedure. We introduce a novel sequential adaptive action value function for query selection using the multi-armed bandit (MAB) framework which allows us to find a tractable solution. For this adaptive-sequential query selection method, we analytically show: (i) performance improvement in the query selection for a dynamical system, (ii) the conditions where the model outperforms competitors. We also present favorable empirical assessments of the performance for this method, compared to alternative methods, both using Monte Carlo simulations and human-in-the-loop experiments with a brain computer interface (BCI) typing system where the language model provides the prior information.

Predicting Imminent Aggression Onset in Minimally-Verbal Youth with Autism Spectrum Disorder Using Preceding Physiological Signals

We test the hypothesis that changes in preceding physiological arousal can be used to predict imminent aggression proximally before it occurs in youth with autism spectrum disorder (ASD) who are minimally verbal (MV-ASD). We evaluate this hypothesis through statistical analyses performed on physiological biosensor data wirelessly recorded from 20 MV-ASD youth over 69 independent naturalistic observations in a hospital inpatient unit. Using ridge-regularized logistic regression, results demonstrate that, on average, our models are able to predict the onset of aggression 1 minute before it occurs using 3 minutes of prior data with a 0.71 AUC for global, and a 0.84 AUC for person-dependent models.

On Analysis of Active Querying for Recursive State Estimation

In stochastic linear/non-linear active dynamic systems, states are estimated with the evidence through recursive measurements in response to queries of the system about the state to be estimated. Therefore, query selection is essential for such systems to improve state estimation accuracy and time. Query selection is conventionally achieved by minimization of the evidence variance or optimization of various information theoretic objectives. It was shown that optimization of mutual information-based objectives and variance-based objectives arrive at the same solution. However, existing approaches optimize approximations to the intended objectives rather than solving the exact optimization problems. To overcome these shortcomings, we propose an active querying procedure using mutual information maximization in recursive state estimation. First we show that mutual information generalizes variance based query selection methods and show the equivalence between objectives if the evidence likelihoods have unimodal distributions. We then solve the exact optimization problem for query selection and propose a query (measurement) selection algorithm. We specifically formulate the mutual information maximization for query selection as a combinatorial optimization problem and show that the objective is sub-modular, therefore can be solved efficiently with guaranteed convergence bounds through a greedy approach. Additionally, we analyze the performance of the query selection algorithm by testing it through a brain computer interface typing system.

INCORPORATING TEMPORAL DEPENDENCY ON ERP BASED BCI

In brain computer interface (BCI) systems based on event related potentials (ERPs), a windowed electroencephalography (EEG) signal is taken into consideration for the assumed duration of the ERP potential. In BCI applications inter stimuli interval is shorter than the ERP duration. This causes temporal dependencies over observation potentials thus disallows taking the data into consideration independently. However, conventionally the data is assumed to be independent for decreasing complexity. In this paper we propose a graphical model which covers the temporal dependency into consideration by labeling each time sample. We also propose a formulation to exploit the time series structure of the EEG.

IMPROVED CLASSIFICATION IN TACTILE BCIS USING A NOISY LABEL MODEL

Tactile BCIs have gained recent popularity in the BCI community due to the advantages of using a stimulation medium which does not inhibit the users visual or auditory senses, is naturally inconspicuous, and can still be used by a person who may be visually or auditorily impaired. While many systems have been proposed which utilize the P300 response elicited through an oddball task, these systems struggle to classify user responses with accuracies comparable to many visual stimulus based systems. In this study, we model the tactile ERP generation as label noise and develop a novel BCI paradigm for binary communication designed to minimize label confusion. The classification model is based on a modified Gaussian mixture and trained using expectation maximization (EM). Finally, we show after testing on multiple subjects that this approach yields cross-validated accuracies for all users which are significantly above chance and suggests that such an approach is robust and reliable for a variety of binary communication-based applications.

INFORMATION THEORETIC FEATURE PROJECTION FOR SINGLE-TRIAL BRAIN-COMPUTER INTERFACES

Current approaches on optimal spatio-spectral feature extraction for single-trial BCIs exploit mutual information based feature ranking and selection algorithms. In order to overcome potential confounders underlying feature selection by information theoretic criteria, we propose a non-parametric feature projection framework for dimensionality reduction that utilizes mutual information based stochastic gradient descent. We demonstrate the feasibility of the protocol based on analyses of EEG data collected during execution of open and close palm hand gestures. We further discuss the approach in terms of potential insights in the context of neurophysiologically driven prosthetic hand control.

Muscle Synergy-based Grasp Classification for Robotic Hand Prosthetics

The main goal of this study is analyzing whether muscle synergies based on surface electromyography (EMG) measurements could be used for hand posture classification in the context of robotic prosthetic control. Target grasps were selected according to usefulness in daily activities. Additionally, due to the feasibility constraints of robotic prosthetics, only 14 gestures (13 feasible grasps and 1 resting state) were analyzed. EMG signals of intact-limb subjects were decomposed into base and activation components for muscle activity evaluation. The results demonstrate that features based on muscle synergies derived from non-negative matrix factorization (NMF) outperform the ones derived from principal component analysis (PCA). Moreover, we also examine the robustness of these methods in the absence of electrodes (muscle importance) and show that NMF is able to provide sufficiently accurate results.