1
|
Cinel C, Fernandez-Vargas J, Tremmel C, Citi L, Poli R. Enhancing performance with multisensory cues in a realistic target discrimination task. PLoS One 2022; 17:e0272320. [PMID: 35930533 PMCID: PMC9355224 DOI: 10.1371/journal.pone.0272320] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2021] [Accepted: 07/17/2022] [Indexed: 11/19/2022] Open
Abstract
Making decisions is an important aspect of people’s lives. Decisions can be highly critical in nature, with mistakes possibly resulting in extremely adverse consequences. Yet, such decisions have often to be made within a very short period of time and with limited information. This can result in decreased accuracy and efficiency. In this paper, we explore the possibility of increasing speed and accuracy of users engaged in the discrimination of realistic targets presented for a very short time, in the presence of unimodal or bimodal cues. More specifically, we present results from an experiment where users were asked to discriminate between targets rapidly appearing in an indoor environment. Unimodal (auditory) or bimodal (audio-visual) cues could shortly precede the target stimulus, warning the users about its location. Our findings show that, when used to facilitate perceptual decision under time pressure, and in condition of limited information in real-world scenarios, spoken cues can be effective in boosting performance (accuracy, reaction times or both), and even more so when presented in bimodal form. However, we also found that cue timing plays a critical role and, if the cue-stimulus interval is too short, cues may offer no advantage. In a post-hoc analysis of our data, we also show that congruency between the response location and both the target location and the cues, can interfere with the speed and accuracy in the task. These effects should be taken in consideration, particularly when investigating performance in realistic tasks.
Collapse
Affiliation(s)
- Caterina Cinel
- Brain Computer Interface and Neural Engineering Lab, School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom
- * E-mail:
| | - Jacobo Fernandez-Vargas
- Brain Computer Interface and Neural Engineering Lab, School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom
| | - Christoph Tremmel
- Brain Computer Interface and Neural Engineering Lab, School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom
- WellthLab, Electronics and Computer Science, University of Southampton, Southampton, United Kingdom
| | - Luca Citi
- Brain Computer Interface and Neural Engineering Lab, School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom
| | - Riccardo Poli
- Brain Computer Interface and Neural Engineering Lab, School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom
| |
Collapse
|
2
|
Tremmel C, Fernandez-Vargas J, Stamos D, Cinel C, Pontil M, Citi L, Poli R. A meta-learning BCI for estimating decision confidence. J Neural Eng 2022; 19. [PMID: 35738232 DOI: 10.1088/1741-2552/ac7ba8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2022] [Accepted: 06/23/2022] [Indexed: 11/12/2022]
Abstract
OBJECTIVE We investigated whether a recently introduced transfer-learning technique based on meta-learning could improve the performance of Brain-Computer Interfaces (BCIs) for decision-confidence prediction with respect to more traditional machine learning methods. APPROACH We adapted the meta-learning by biased regularisation algorithm to the problem of predicting decision confidence from EEG and EOG data on a decision-by-decision basis in a difficult target discrimination task based on video feeds. The method exploits previous participants' data to produce a prediction algorithm that is then quickly tuned to new participants. We compared it with with the traditional single-subject training almost universally adopted in BCIs, a state-of-the-art transfer learning technique called Domain Adversarial Neural Networks (DANN), a transfer-learning adaptation of a zero-training method we used recently for a similar task, and with a simple baseline algorithm. MAIN RESULTS The meta-learning approach was significantly better than other approaches in most conditions, and much better in situations where limited data from a new participant are available for training/tuning. Meta-learning by biased regularisation allowed our BCI to seamlessly integrate information from past participants with data from a specific user to produce high-performance predictors. Its robustness in the presence of small training sets is a real-plus in BCI applications, as new users need to train the BCI for a much shorter period. SIGNIFICANCE Due to the variability and noise of EEG/EOG data, BCIs need to be normally trained with data from a specific participant. This work shows that even better performance can be obtained using our version of meta-learning by biased regularisation.
Collapse
Affiliation(s)
- Christoph Tremmel
- School of Computer Science and Electronic Engineering, University of Essex, Wivenhoe Park, Colchester, Essex, CO4 3SQ, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Jacobo Fernandez-Vargas
- School of Computer Science and Electronic Engineering, University of Essex, Wivenhoe Park, Colchester, Essex, CO4 3SQ, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Dimitrios Stamos
- Department of Computer Science, University College London, Malet Place, London, London, WC1E 6BT, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Caterina Cinel
- School of Computer Science and Electronic Engineering, University of Essex, Wivenhoe Park, Colchester, Essex, CO4 3SQ, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Massimiliano Pontil
- University College London, Malet Place, London, London, WC1E 6BT, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Luca Citi
- School of Computer Science and Electronic Engineering, University of Essex, Wivenhoe Park, Colchester, Essex, CO4 3SQ, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Riccardo Poli
- School of Computer Science and Electronic Engineering, University of Essex, Wivenhoe Park, Colchester, Essex, CO4 3SQ, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| |
Collapse
|
3
|
Fernandez-Vargas J, Tremmel C, Valeriani D, Bhattacharyya S, Cinel C, Citi L, Poli R. Subject- and task-independent neural correlates and prediction of decision confidence in perceptual decision making. J Neural Eng 2021; 18. [PMID: 33780913 DOI: 10.1088/1741-2552/abf2e4] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2020] [Accepted: 03/29/2021] [Indexed: 11/12/2022]
Abstract
Objective.In many real-world decision tasks, the information available to the decision maker is incomplete. To account for this uncertainty, we associate a degree of confidence to every decision, representing the likelihood of that decision being correct. In this study, we analyse electroencephalography (EEG) data from 68 participants undertaking eight different perceptual decision-making experiments. Our goals are to investigate (1) whether subject- and task-independent neural correlates of decision confidence exist, and (2) to what degree it is possible to build brain computer interfaces that can estimate confidence on a trial-by-trial basis. The experiments cover a wide range of perceptual tasks, which allowed to separate the task-related, decision-making features from the task-independent ones.Approach.Our systems train artificial neural networks to predict the confidence in each decision from EEG data and response times. We compare the decoding performance with three training approaches: (1) single subject, where both training and testing data were acquired from the same person; (2) multi-subject, where all the data pertained to the same task, but the training and testing data came from different users; and (3) multi-task, where the training and testing data came from different tasks and subjects. Finally, we validated our multi-task approach using data from two additional experiments, in which confidence was not reported.Main results.We found significant differences in the EEG data for different confidence levels in both stimulus-locked and response-locked epochs. All our approaches were able to predict the confidence between 15% and 35% better than the corresponding reference baselines.Significance.Our results suggest that confidence in perceptual decision making tasks could be reconstructed from neural signals even when using transfer learning approaches. These confidence estimates are based on the decision-making process rather than just the confidence-reporting process.
Collapse
Affiliation(s)
- Jacobo Fernandez-Vargas
- Brain-Computer Interfaces and Neural Engineering laboratory, School of Computer Science and Electronic Engineering, University of Essex, Essex, United Kingdom
| | - Christoph Tremmel
- Brain-Computer Interfaces and Neural Engineering laboratory, School of Computer Science and Electronic Engineering, University of Essex, Essex, United Kingdom
| | - Davide Valeriani
- Department of Otolaryngology
- Head and Neck Surgery, Massachusetts Eye and Ear, Boston, MA, United States of America.,Department of Otolaryngology
- Head and Neck Surgery, Harvard Medical School, Boston, MA, United States of America
| | - Saugat Bhattacharyya
- Brain-Computer Interfaces and Neural Engineering laboratory, School of Computer Science and Electronic Engineering, University of Essex, Essex, United Kingdom.,School of Computing, Engineering & Intelligent Systems, Ulster University, Londonderry, United Kingdom
| | - Caterina Cinel
- Brain-Computer Interfaces and Neural Engineering laboratory, School of Computer Science and Electronic Engineering, University of Essex, Essex, United Kingdom
| | - Luca Citi
- Brain-Computer Interfaces and Neural Engineering laboratory, School of Computer Science and Electronic Engineering, University of Essex, Essex, United Kingdom
| | - Riccardo Poli
- Brain-Computer Interfaces and Neural Engineering laboratory, School of Computer Science and Electronic Engineering, University of Essex, Essex, United Kingdom
| |
Collapse
|
4
|
Fernandez-Vargas J, Kita K, Yu W. Real-time Hand Motion Reconstruction System for Trans-Humeral Amputees Using EEG and EMG. Front Robot AI 2016. [DOI: 10.3389/frobt.2016.00050] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/28/2023] Open
|
5
|
Fernandez-Vargas J, Pfaff HU, Rodríguez FB, Varona P. Assisted closed-loops for brain-computer interfaces. BMC Neurosci 2013. [PMCID: PMC3704705 DOI: 10.1186/1471-2202-14-s1-p406] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022] Open
|
6
|
Fernandez-Vargas J, Pfaff HU, Rodríguez FB, Varona P. Assisted closed-loop optimization of SSVEP-BCI efficiency. Front Neural Circuits 2013; 7:27. [PMID: 23443214 PMCID: PMC3580891 DOI: 10.3389/fncir.2013.00027] [Citation(s) in RCA: 28] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2012] [Accepted: 02/06/2013] [Indexed: 11/23/2022] Open
Abstract
We designed a novel assisted closed-loop optimization protocol to improve the efficiency of brain-computer interfaces (BCI) based on steady state visually evoked potentials (SSVEP). In traditional paradigms, the control over the BCI-performance completely depends on the subjects' ability to learn from the given feedback cues. By contrast, in the proposed protocol both the subject and the machine share information and control over the BCI goal. Generally, the innovative assistance consists in the delivery of online information together with the online adaptation of BCI stimuli properties. In our case, this adaptive optimization process is realized by (1) a closed-loop search for the best set of SSVEP flicker frequencies and (2) feedback of actual SSVEP magnitudes to both the subject and the machine. These closed-loop interactions between subject and machine are evaluated in real-time by continuous measurement of their efficiencies, which are used as online criteria to adapt the BCI control parameters. The proposed protocol aims to compensate for variability in possibly unknown subjects' state and trait dimensions. In a study with N = 18 subjects, we found significant evidence that our protocol outperformed classic SSVEP-BCI control paradigms. Evidence is presented that it takes indeed into account interindividual variabilities: e.g., under the new protocol, baseline resting state EEG measures predict subjects' BCI performances. This paper illustrates the promising potential of assisted closed-loop protocols in BCI systems. Probably their applicability might be expanded to innovative uses, e.g., as possible new diagnostic/therapeutic tools for clinical contexts and as new paradigms for basic research.
Collapse
Affiliation(s)
- Jacobo Fernandez-Vargas
- Grupo de Neurocomputación Biológica, Departamento de Ingeniería Informática, Escuela Politécnica Superior, Universidad Autónoma de Madrid Madrid, Spain
| | | | | | | |
Collapse
|