1
|
Kandemir G, Wilhelm SA, Axmacher N, Akyürek EG. Maintenance of color memoranda in activity-quiescent working memory states: Evidence from impulse perturbation. iScience 2024; 27:109565. [PMID: 38617556 PMCID: PMC11015458 DOI: 10.1016/j.isci.2024.109565] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2023] [Revised: 01/22/2024] [Accepted: 03/22/2024] [Indexed: 04/16/2024] Open
Abstract
In the present study, we used an impulse perturbation method to probe working memory maintenance of colors in neurally active and activity-quiescent states, focusing on a set of pre-registered analyses. We analyzed the electroencephalograph (EEG) data of 30 participants who completed a delayed match-to-sample working memory task, in which one of the two items that were presented was retro-cued as task relevant. The analyses revealed that both cued and uncued colors were decodable from impulse-evoked activity, the latter in contrast to previous reports of working memory for orientation gratings. Decoding of colors from oscillations in the alpha band showed that cued items could be decoded therein whereas uncued items could not. Overall, the outcomes suggest that subtle differences exist between the representation of colors, and that of stimuli with spatial properties, but the present results also demonstrate that regardless of their specific neural state, both are accessible through visual impulse perturbation.
Collapse
Affiliation(s)
- Güven Kandemir
- Department of Experimental Psychology, University of Groningen, Groningen 9712 TS, the Netherlands
- Department of Experimental and Applied Psychology, Vrije Universiteit Amsterdam, Amsterdam 1081 BT, the Netherlands
| | - Sophia A. Wilhelm
- Department of Experimental Psychology, University of Groningen, Groningen 9712 TS, the Netherlands
| | - Nikolai Axmacher
- Department of Neuropsychology, Faculty of Psychology, Ruhr University Bochum, 44780 Bochum, Germany
| | - Elkan G. Akyürek
- Department of Experimental Psychology, University of Groningen, Groningen 9712 TS, the Netherlands
| |
Collapse
|
2
|
Linde-Domingo J, Spitzer B. Geometry of visuospatial working memory information in miniature gaze patterns. Nat Hum Behav 2024; 8:336-348. [PMID: 38110511 PMCID: PMC10896725 DOI: 10.1038/s41562-023-01737-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Accepted: 09/25/2023] [Indexed: 12/20/2023]
Abstract
Stimulus-dependent eye movements have been recognized as a potential confound in decoding visual working memory information from neural signals. Here we combined eye-tracking with representational geometry analyses to uncover the information in miniature gaze patterns while participants (n = 41) were cued to maintain visual object orientations. Although participants were discouraged from breaking fixation by means of real-time feedback, small gaze shifts (<1°) robustly encoded the to-be-maintained stimulus orientation, with evidence for encoding two sequentially presented orientations at the same time. The orientation encoding on stimulus presentation was object-specific, but it changed to a more object-independent format during cued maintenance, particularly when attention had been temporarily withdrawn from the memorandum. Finally, categorical reporting biases increased after unattended storage, with indications of biased gaze geometries already emerging during the maintenance periods before behavioural reporting. These findings disclose a wealth of information in gaze patterns during visuospatial working memory and indicate systematic changes in representational format when memory contents have been unattended.
Collapse
Affiliation(s)
- Juan Linde-Domingo
- Research Group Adaptive Memory and Decision Making, Max Planck Institute for Human Development, Berlin, Germany.
- Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany.
- Mind, Brain and Behavior Research Center, University of Granada, Granada, Spain.
- Department of Experimental Psychology, University of Granada, Granada, Spain.
| | - Bernhard Spitzer
- Research Group Adaptive Memory and Decision Making, Max Planck Institute for Human Development, Berlin, Germany.
- Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany.
| |
Collapse
|
3
|
de Vries E, van Ede F. Microsaccades Track Location-Based Object Rehearsal in Visual Working Memory. eNeuro 2024; 11:ENEURO.0276-23.2023. [PMID: 38176905 PMCID: PMC10849020 DOI: 10.1523/eneuro.0276-23.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Revised: 12/05/2023] [Accepted: 12/09/2023] [Indexed: 01/06/2024] Open
Abstract
Besides controlling eye movements, the brain's oculomotor system has been implicated in the control of covert spatial attention and the rehearsal of spatial information in working memory. We investigated whether the oculomotor system also contributes to rehearsing visual objects in working memory when object location is never asked about. To address this, we tracked the incidental use of locations for mnemonic rehearsal via directional biases in microsaccades while participants maintained two visual objects (colored oriented gratings) in working memory. By varying the stimulus configuration (horizontal, diagonal, and vertical) at encoding, we could quantify whether microsaccades were more aligned with the configurational axis of the memory contents, as opposed to the orthogonal axis. Experiment 1 revealed that microsaccades continued to be biased along the axis of the memory content several seconds into the working memory delay. In Experiment 2, we confirmed that this directional microsaccade bias was specific to memory demands, ruling out lingering effects from passive and attentive encoding of the same visual objects in the same configurations. Thus, by studying microsaccade directions, we uncover oculomotor-driven rehearsal of visual objects in working memory through their associated locations.
Collapse
Affiliation(s)
- Eelke de Vries
- Department of Experimental and Applied Psychology, Institute for Brain and Behavior Amsterdam, Vrije Universiteit Amsterdam, Amsterdam 1081 HV, The Netherlands
| | - Freek van Ede
- Department of Experimental and Applied Psychology, Institute for Brain and Behavior Amsterdam, Vrije Universiteit Amsterdam, Amsterdam 1081 HV, The Netherlands
| |
Collapse
|
4
|
Barbieri R, Töpfer FM, Soch J, Bogler C, Sprekeler H, Haynes JD. Encoding of continuous perceptual choices in human early visual cortex. Front Hum Neurosci 2023; 17:1277539. [PMID: 38021249 PMCID: PMC10679739 DOI: 10.3389/fnhum.2023.1277539] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Accepted: 10/25/2023] [Indexed: 12/01/2023] Open
Abstract
Introduction Research on the neural mechanisms of perceptual decision-making has typically focused on simple categorical choices, say between two alternative motion directions. Studies on such discrete alternatives have often suggested that choices are encoded either in a motor-based or in an abstract, categorical format in regions beyond sensory cortex. Methods In this study, we used motion stimuli that could vary anywhere between 0° and 360° to assess how the brain encodes choices for features that span the full sensory continuum. We employed a combination of neuroimaging and encoding models based on Gaussian process regression to assess how either stimuli or choices were encoded in brain responses. Results We found that single-voxel tuning patterns could be used to reconstruct the trial-by-trial physical direction of motion as well as the participants' continuous choices. Importantly, these continuous choice signals were primarily observed in early visual areas. The tuning properties in this region generalized between choice encoding and stimulus encoding, even for reports that reflected pure guessing. Discussion We found only little information related to the decision outcome in regions beyond visual cortex, such as parietal cortex, possibly because our task did not involve differential motor preparation. This could suggest that decisions for continuous stimuli take can place already in sensory brain regions, potentially using similar mechanisms to the sensory recruitment in visual working memory.
Collapse
Affiliation(s)
- Riccardo Barbieri
- Bernstein Center for Computational Neuroscience and Berlin Center for Advanced Neuroimaging, Department of Neurology, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin and Berlin Institute of Health (BIH), Berlin, Germany
| | - Felix M. Töpfer
- Bernstein Center for Computational Neuroscience and Berlin Center for Advanced Neuroimaging, Department of Neurology, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin and Berlin Institute of Health (BIH), Berlin, Germany
| | - Joram Soch
- Bernstein Center for Computational Neuroscience and Berlin Center for Advanced Neuroimaging, Department of Neurology, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin and Berlin Institute of Health (BIH), Berlin, Germany
- German Center for Neurodegenerative Diseases, Göttingen, Germany
| | - Carsten Bogler
- Bernstein Center for Computational Neuroscience and Berlin Center for Advanced Neuroimaging, Department of Neurology, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin and Berlin Institute of Health (BIH), Berlin, Germany
| | - Henning Sprekeler
- Department for Electrical Engineering and Computer Science, Technische Universität Berlin, Berlin, Germany
| | - John-Dylan Haynes
- Bernstein Center for Computational Neuroscience and Berlin Center for Advanced Neuroimaging, Department of Neurology, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin and Berlin Institute of Health (BIH), Berlin, Germany
- Berlin School of Mind and Brain and Institute of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
| |
Collapse
|
5
|
Mari T, Henderson J, Ali SH, Hewitt D, Brown C, Stancak A, Fallon N. Machine learning and EEG can classify passive viewing of discrete categories of visual stimuli but not the observation of pain. BMC Neurosci 2023; 24:50. [PMID: 37715119 PMCID: PMC10504739 DOI: 10.1186/s12868-023-00819-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2023] [Accepted: 09/06/2023] [Indexed: 09/17/2023] Open
Abstract
Previous studies have demonstrated the potential of machine learning (ML) in classifying physical pain from non-pain states using electroencephalographic (EEG) data. However, the application of ML to EEG data to categorise the observation of pain versus non-pain images of human facial expressions or scenes depicting pain being inflicted has not been explored. The present study aimed to address this by training Random Forest (RF) models on cortical event-related potentials (ERPs) recorded while participants passively viewed faces displaying either pain or neutral expressions, as well as action scenes depicting pain or matched non-pain (neutral) scenarios. Ninety-one participants were recruited across three samples, which included a model development group (n = 40) and a cross-subject validation group (n = 51). Additionally, 25 participants from the model development group completed a second experimental session, providing a within-subject temporal validation sample. The analysis of ERPs revealed an enhanced N170 component in response to faces compared to action scenes. Moreover, an increased late positive potential (LPP) was observed during the viewing of pain scenes compared to neutral scenes. Additionally, an enhanced P3 response was found when participants viewed faces displaying pain expressions compared to neutral expressions. Subsequently, three RF models were developed to classify images into faces and scenes, neutral and pain scenes, and neutral and pain expressions. The RF model achieved classification accuracies of 75%, 64%, and 69% for cross-validation, cross-subject, and within-subject classifications, respectively, along with reasonably calibrated predictions for the classification of face versus scene images. However, the RF model was unable to classify pain versus neutral stimuli above chance levels when presented with subsequent tasks involving images from either category. These results expand upon previous findings by externally validating the use of ML in classifying ERPs related to different categories of visual images, namely faces and scenes. The results also indicate the limitations of ML in distinguishing pain and non-pain connotations using ERP responses to the passive viewing of visually similar images.
Collapse
Affiliation(s)
- Tyler Mari
- Department of Psychology, Institute of Population Health, University of Liverpool, 2.21 Eleanor Rathbone Building, Bedford Street South, Liverpool, L69 7ZA, UK.
| | - Jessica Henderson
- Department of Psychology, Institute of Population Health, University of Liverpool, 2.21 Eleanor Rathbone Building, Bedford Street South, Liverpool, L69 7ZA, UK
| | - S Hasan Ali
- Department of Psychology, Institute of Population Health, University of Liverpool, 2.21 Eleanor Rathbone Building, Bedford Street South, Liverpool, L69 7ZA, UK
| | - Danielle Hewitt
- Department of Psychology, Institute of Population Health, University of Liverpool, 2.21 Eleanor Rathbone Building, Bedford Street South, Liverpool, L69 7ZA, UK
| | - Christopher Brown
- Department of Psychology, Institute of Population Health, University of Liverpool, 2.21 Eleanor Rathbone Building, Bedford Street South, Liverpool, L69 7ZA, UK
| | - Andrej Stancak
- Department of Psychology, Institute of Population Health, University of Liverpool, 2.21 Eleanor Rathbone Building, Bedford Street South, Liverpool, L69 7ZA, UK
| | - Nicholas Fallon
- Department of Psychology, Institute of Population Health, University of Liverpool, 2.21 Eleanor Rathbone Building, Bedford Street South, Liverpool, L69 7ZA, UK
| |
Collapse
|
6
|
Peelen MV, Downing PE. Testing cognitive theories with multivariate pattern analysis of neuroimaging data. Nat Hum Behav 2023; 7:1430-1441. [PMID: 37591984 DOI: 10.1038/s41562-023-01680-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2022] [Accepted: 07/12/2023] [Indexed: 08/19/2023]
Abstract
Multivariate pattern analysis (MVPA) has emerged as a powerful method for the analysis of functional magnetic resonance imaging, electroencephalography and magnetoencephalography data. The new approaches to experimental design and hypothesis testing afforded by MVPA have made it possible to address theories that describe cognition at the functional level. Here we review a selection of studies that have used MVPA to test cognitive theories from a range of domains, including perception, attention, memory, navigation, emotion, social cognition and motor control. This broad view reveals properties of MVPA that make it suitable for understanding the 'how' of human cognition, such as the ability to test predictions expressed at the item or event level. It also reveals limitations and points to future directions.
Collapse
Affiliation(s)
- Marius V Peelen
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands.
| | - Paul E Downing
- Cognitive Neuroscience Institute, Department of Psychology, Bangor University, Bangor, UK.
| |
Collapse
|
7
|
Liu B, Nobre AC, van Ede F. Microsaccades transiently lateralise EEG alpha activity. Prog Neurobiol 2023; 224:102433. [PMID: 36907349 PMCID: PMC10074474 DOI: 10.1016/j.pneurobio.2023.102433] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2022] [Revised: 03/08/2023] [Accepted: 03/09/2023] [Indexed: 03/12/2023]
Abstract
The lateralisation of 8-12 Hz alpha activity is a canonical signature of human spatial cognition that is typically studied under strict fixation requirements. Yet, even during attempted fixation, the brain produces small involuntary eye movements known as microsaccades. Here we report how spontaneous microsaccades - made in the absence of incentives to look elsewhere - can themselves drive transient lateralisation of EEG alpha power according to microsaccade direction. This transient lateralisation of posterior alpha power occurs similarly following start and return microsaccades and is, at least for start microsaccades, driven by increased alpha power ipsilateral to microsaccade direction. This reveals new links between spontaneous microsaccades and human electrophysiological brain activity. It highlights how microsaccades are an important factor to consider in studies relating alpha activity - including spontaneous fluctuations in alpha activity - to spatial cognition, such as studies on visual attention, anticipation, and working memory.
Collapse
Affiliation(s)
- Baiwei Liu
- Institute for Brain and Behavior Amsterdam, Department of Experimental and Applied Psychology, Vrije Universiteit Amsterdam, the Netherlands.
| | - Anna C Nobre
- Department of Experimental Psychology, University of Oxford, United Kingdom; Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, United Kingdom
| | - Freek van Ede
- Institute for Brain and Behavior Amsterdam, Department of Experimental and Applied Psychology, Vrije Universiteit Amsterdam, the Netherlands; Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, United Kingdom.
| |
Collapse
|
8
|
Li R, Hu H, Zhao X, Wang Z, Xu G. A static paradigm based on illusion-induced VEP for brain-computer interfaces. J Neural Eng 2023; 20:026006. [PMID: 36808912 DOI: 10.1088/1741-2552/acbdc0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/23/2023]
Abstract
OBJECTIVE Visual evoked potentials (VEPs) have been commonly applied in brain-computer interfaces (BCIs) due to their satisfactory classification performance recently. However, most existing methods with flickering or oscillating stimuli will induce visual fatigue under long-term training, thus restricting the implementation of VEP-based BCIs. To address this issue, a novel paradigm adopting static motion illusion based on illusion-induced visual evoked potential (IVEP) is proposed for BCIs to enhance visual experience and practicality. APPROACH This study explored the responses to baseline and illusion tasks including the Rotating-Tilted-Lines (RTL) illusion and Rotating-Snakes (RS) illusion. The distinguishable features were examined between different illusions by analyzing the event-related potentials (ERPs) and amplitude modulation of evoked oscillatory responses. MAIN RESULTS The illusion stimuli elicited VEPs in an early time window encompassing a negative component (N1) from 110 to 200 ms and a positive component (P2) between 210 and 300 ms. Based on the feature analysis, a filter bank was designed to extract discriminative signals. The task-related component analysis (TRCA) was used to evaluate the binary classification task performance of the proposed method. Then the highest accuracy of 86.67% was achieved with a data length of 0.6 s. SIGNIFICANCE The results of this study demonstrate that the static motion illusion paradigm has the feasibility of implementation and is promising for VEP-based BCI applications.
Collapse
Affiliation(s)
- Ruxue Li
- Intelligent Information and Communication Technology Research and Development Center, Shanghai Advanced Research Institute Chinese Academy of Sciences, 99 Haike Road, Pudong New Area, Shanghai, Shanghai, 201210, CHINA
| | - Honglin Hu
- Intelligent Information and Communication Technology Research and Development Center, Shanghai Advanced Research Institute Chinese Academy of Sciences, 99 Haike Road, Pudong New Area, Shanghai, Shanghai, 201210, CHINA
| | - Xi Zhao
- Intelligent Information and Communication Technology Research and Development Center, Shanghai Advanced Research Institute, 99 Haike Road, Pudong New Area, Shanghai, Shanghai, 201210, CHINA
| | - Zhenyu Wang
- Intelligent Information and Communication Technology Research and Development Center, Shanghai Advanced Research Institute Chinese Academy of Sciences, 99 Haike Road, Pudong New Area, Shanghai, Shanghai, 201210, CHINA
| | - Guiying Xu
- Intelligent Information and Communication Technology Research and Development Center, Shanghai Advanced Research Institute, 99 Haike Road, Pudong New Area, Shanghai, Shanghai, Shanghai, 201210, CHINA
| |
Collapse
|
9
|
The Spatiotemporal Neural Dynamics of Object Recognition for Natural Images and Line Drawings. J Neurosci 2023; 43:484-500. [PMID: 36535769 PMCID: PMC9864561 DOI: 10.1523/jneurosci.1546-22.2022] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2022] [Revised: 11/18/2022] [Accepted: 11/30/2022] [Indexed: 12/24/2022] Open
Abstract
Drawings offer a simple and efficient way to communicate meaning. While line drawings capture only coarsely how objects look in reality, we still perceive them as resembling real-world objects. Previous work has shown that this perceived similarity is mirrored by shared neural representations for drawings and natural images, which suggests that similar mechanisms underlie the recognition of both. However, other work has proposed that representations of drawings and natural images become similar only after substantial processing has taken place, suggesting distinct mechanisms. To arbitrate between those alternatives, we measured brain responses resolved in space and time using fMRI and MEG, respectively, while human participants (female and male) viewed images of objects depicted as photographs, line drawings, or sketch-like drawings. Using multivariate decoding, we demonstrate that object category information emerged similarly fast and across overlapping regions in occipital, ventral-temporal, and posterior parietal cortex for all types of depiction, yet with smaller effects at higher levels of visual abstraction. In addition, cross-decoding between depiction types revealed strong generalization of object category information from early processing stages on. Finally, by combining fMRI and MEG data using representational similarity analysis, we found that visual information traversed similar processing stages for all types of depiction, yet with an overall stronger representation for photographs. Together, our results demonstrate broad commonalities in the neural dynamics of object recognition across types of depiction, thus providing clear evidence for shared neural mechanisms underlying recognition of natural object images and abstract drawings.SIGNIFICANCE STATEMENT When we see a line drawing, we effortlessly recognize it as an object in the world despite its simple and abstract style. Here we asked to what extent this correspondence in perception is reflected in the brain. To answer this question, we measured how neural processing of objects depicted as photographs and line drawings with varying levels of detail (from natural images to abstract line drawings) evolves over space and time. We find broad commonalities in the spatiotemporal dynamics and the neural representations underlying the perception of photographs and even abstract drawings. These results indicate a shared basic mechanism supporting recognition of drawings and natural images.
Collapse
|
10
|
Fullerton AM, Vickers DA, Luke R, Billing AN, McAlpine D, Hernandez-Perez H, Peelle JE, Monaghan JJM, McMahon CM. Cross-modal functional connectivity supports speech understanding in cochlear implant users. Cereb Cortex 2022; 33:3350-3371. [PMID: 35989307 PMCID: PMC10068270 DOI: 10.1093/cercor/bhac277] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2021] [Revised: 06/10/2022] [Accepted: 06/11/2022] [Indexed: 11/12/2022] Open
Abstract
Sensory deprivation can lead to cross-modal cortical changes, whereby sensory brain regions deprived of input may be recruited to perform atypical function. Enhanced cross-modal responses to visual stimuli observed in auditory cortex of postlingually deaf cochlear implant (CI) users are hypothesized to reflect increased activation of cortical language regions, but it is unclear if this cross-modal activity is "adaptive" or "mal-adaptive" for speech understanding. To determine if increased activation of language regions is correlated with better speech understanding in CI users, we assessed task-related activation and functional connectivity of auditory and visual cortices to auditory and visual speech and non-speech stimuli in CI users (n = 14) and normal-hearing listeners (n = 17) and used functional near-infrared spectroscopy to measure hemodynamic responses. We used visually presented speech and non-speech to investigate neural processes related to linguistic content and observed that CI users show beneficial cross-modal effects. Specifically, an increase in connectivity between the left auditory and visual cortices-presumed primary sites of cortical language processing-was positively correlated with CI users' abilities to understand speech in background noise. Cross-modal activity in auditory cortex of postlingually deaf CI users may reflect adaptive activity of a distributed, multimodal speech network, recruited to enhance speech understanding.
Collapse
Affiliation(s)
- Amanda M Fullerton
- Department of Linguistics and Macquarie University Hearing, Australian Hearing Hub, Macquarie University, Sydney 2109, Australia
| | - Deborah A Vickers
- Cambridge Hearing Group, Sound Lab, Department of Clinical Neurosciences, University of Cambridge, Cambridge CB2 OSZ, United Kingdom.,Speech, Hearing and Phonetic Sciences, University College London, London WC1N 1PF, United Kingdom
| | - Robert Luke
- Department of Linguistics and Macquarie University Hearing, Australian Hearing Hub, Macquarie University, Sydney 2109, Australia
| | - Addison N Billing
- Institute of Cognitive Neuroscience, University College London, London WCIN 3AZ, United Kingdom.,DOT-HUB, Department of Medical Physics and Biomedical Engineering, University College London, London WC1E 6BT, United Kingdom
| | - David McAlpine
- Department of Linguistics and Macquarie University Hearing, Australian Hearing Hub, Macquarie University, Sydney 2109, Australia
| | - Heivet Hernandez-Perez
- Department of Linguistics and Macquarie University Hearing, Australian Hearing Hub, Macquarie University, Sydney 2109, Australia
| | - Jonathan E Peelle
- Department of Otolaryngology, Washington University in St. Louis, St. Louis, MO 63110, United States
| | - Jessica J M Monaghan
- National Acoustic Laboratories, Australian Hearing Hub, Sydney 2109, Australia.,Department of Linguistics and Macquarie University Hearing, Australian Hearing Hub, Macquarie University, Sydney 2109, Australia
| | - Catherine M McMahon
- Department of Linguistics and Macquarie University Hearing, Australian Hearing Hub, Macquarie University, Sydney 2109, Australia.,HEAR Centre, Macquarie University, Sydney 2109, Australia
| |
Collapse
|
11
|
van Es MWJ, Marshall TR, Spaak E, Jensen O, Schoffelen J. Phasic modulation of visual representations during sustained attention. Eur J Neurosci 2022; 55:3191-3208. [PMID: 33319447 PMCID: PMC9543919 DOI: 10.1111/ejn.15084] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2020] [Revised: 12/09/2020] [Accepted: 12/09/2020] [Indexed: 11/27/2022]
Abstract
Sustained attention has long been thought to benefit perception in a continuous fashion, but recent evidence suggests that it affects perception in a discrete, rhythmic way. Periodic fluctuations in behavioral performance over time, and modulations of behavioral performance by the phase of spontaneous oscillatory brain activity point to an attentional sampling rate in the theta or alpha frequency range. We investigated whether such discrete sampling by attention is reflected in periodic fluctuations in the decodability of visual stimulus orientation from magnetoencephalographic (MEG) brain signals. In this exploratory study, human subjects attended one of the two grating stimuli, while MEG was being recorded. We assessed the strength of the visual representation of the attended stimulus using a support vector machine (SVM) to decode the orientation of the grating (clockwise vs. counterclockwise) from the MEG signal. We tested whether decoder performance depended on the theta/alpha phase of local brain activity. While the phase of ongoing activity in the visual cortex did not modulate decoding performance, theta/alpha phase of activity in the frontal eye fields and parietal cortex, contralateral to the attended stimulus did modulate decoding performance. These findings suggest that phasic modulations of visual stimulus representations in the brain are caused by frequency-specific top-down activity in the frontoparietal attention network, though the behavioral relevance of these effects could not be established.
Collapse
Affiliation(s)
- Mats W. J. van Es
- Donders Institute for Brain, Cognition and BehaviourRadboud University NijmegenNijmegenThe Netherlands
- Wellcome Centre for Integrative NeuroimagingUniversity of OxfordOxfordUK
| | - Tom R. Marshall
- Donders Institute for Brain, Cognition and BehaviourRadboud University NijmegenNijmegenThe Netherlands
- Wellcome Centre for Integrative NeuroimagingUniversity of OxfordOxfordUK
| | - Eelke Spaak
- Donders Institute for Brain, Cognition and BehaviourRadboud University NijmegenNijmegenThe Netherlands
| | - Ole Jensen
- Donders Institute for Brain, Cognition and BehaviourRadboud University NijmegenNijmegenThe Netherlands
- Department of PsychologyUniversity of BirminghamBirminghamUK
| | - Jan‐Mathijs Schoffelen
- Donders Institute for Brain, Cognition and BehaviourRadboud University NijmegenNijmegenThe Netherlands
| |
Collapse
|
12
|
Welke D, Vessel EA. Naturalistic viewing conditions can increase task engagement and aesthetic preference but have only minimal impact on EEG quality. Neuroimage 2022; 256:119218. [PMID: 35443219 DOI: 10.1016/j.neuroimage.2022.119218] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2021] [Revised: 04/06/2022] [Accepted: 04/14/2022] [Indexed: 10/18/2022] Open
Abstract
Free gaze and moving images are typically avoided in EEG experiments due to the expected generation of artifacts and noise. Yet for a growing number of research questions, loosening these rigorous restrictions would be beneficial. Among these is research on visual aesthetic experiences, which often involve open-ended exploration of highly variable stimuli. Here we systematically compare the effect of conservative vs. more liberal experimental settings on various measures of behavior, brain activity and physiology in an aesthetic rating task. Our primary aim was to assess EEG signal quality. 43 participants either maintained fixation or were allowed to gaze freely, and viewed either static images or dynamic (video) stimuli consisting of dance performances or nature scenes. A passive auditory background task (auditory steady-state response; ASSR) was added as a proxy measure for overall EEG recording quality. We recorded EEG, ECG and eyetracking data, and participants rated their aesthetic preference and state of boredom on each trial. Whereas both behavioral ratings and gaze behavior were affected by task and stimulus manipulations, EEG SNR was barely affected and generally robust across all conditions, despite only minimal preprocessing and no trial rejection. In particular, we show that using video stimuli does not necessarily result in lower EEG quality and can, on the contrary, significantly reduce eye movements while increasing both the participants' aesthetic response and general task engagement. We see these as encouraging results indicating that - at least in the lab - more liberal experimental conditions can be adopted without significant loss of signal quality.
Collapse
Affiliation(s)
- Dominik Welke
- Department of Neuroscience, Max Planck Institute for Empirical Aesthetics, Frankfurt (Main), Germany.
| | - Edward A Vessel
- Department of Neuroscience, Max Planck Institute for Empirical Aesthetics, Frankfurt (Main), Germany.
| |
Collapse
|
13
|
Kim SY, Moon BY, Cho HG, Yu DS. Quantitative Evaluation of the Association Between Fixation Stability and Phoria During Short-Term Binocular Viewing. Front Neurosci 2022; 16:721665. [PMID: 35368249 PMCID: PMC8965591 DOI: 10.3389/fnins.2022.721665] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2021] [Accepted: 02/11/2022] [Indexed: 11/29/2022] Open
Abstract
Purpose Fixation stability for binocular anomalies with a phoria cannot be detected by direct observations. This study aimed to quantitatively evaluate fixation stability using an eye tracker rather than direct directions in binocular vision with abnormal and normal phorias. Methods Thirty-five and 25 participants with abnormal and normal phoria, respectively, were included in the study. The horizontal and vertical gaze points and convergence were recorded for 10 s using a remote eye tracker while binocularly viewing a target on a display screen 550 mm away. Fixation stability was quantified using bivariate contour ellipse areas (BCEA). Results The fixation stability for all participants-based evaluations as a single cluster in the abnormal phoria group was lower than that in the normal phoria group (p = 0.005). There was no difference between the two groups in the evaluation based on the BCEA for each participant-based evaluation (p = 0.66). Fixation stability was also more related to convergence for the abnormal phoria group than for the normal phoria group (r = 0.769, p < 0.001; r = 0.417, p = 0.038, respectively). Conclusion As the first study to evaluate fixation stability using an eye-tracker to differentiate between abnormal and normal phoria for non-strabismus, these findings may provide evidence for improving the evaluation of binocular vision not detected with clinical diagnostic tests.
Collapse
|
14
|
The human primary visual cortex (V1) encodes the perceived position of static but not moving objects. Commun Biol 2022; 5:181. [PMID: 35233067 PMCID: PMC8888673 DOI: 10.1038/s42003-022-03136-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2021] [Accepted: 02/03/2022] [Indexed: 11/09/2022] Open
Abstract
Brain activity in retinotopic cortex reflects illusory changes in stimulus position. Is this neural signature a general code for apparent position? Here we show that responses in primary visual cortex (V1) are consistent with perception of the Muller-Lyer illusion; however, we found no such signature for another striking illusion, the curveball effect. This demonstrates that V1 does not encode apparent position per se.
Collapse
|
15
|
Polychroni N, Herrojo Ruiz M, Terhune DB. Introspection confidence predicts EEG decoding of self-generated thoughts and meta-awareness. Hum Brain Mapp 2022; 43:2311-2327. [PMID: 35122359 PMCID: PMC8996352 DOI: 10.1002/hbm.25789] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Revised: 12/17/2021] [Accepted: 12/29/2021] [Indexed: 01/22/2023] Open
Abstract
The neurophysiological bases of mind wandering (MW)-an experiential state wherein attention is disengaged from the external environment in favour of internal thoughts-and state meta-awareness are poorly understood. In parallel, the relationship between introspection confidence in experiential state judgements and neural representations remains unclear. Here, we recorded EEG while participants completed a listening task within which they made experiential state judgements and rated their confidence. Alpha power was reliably greater during MW episodes, with unaware MW further associated with greater delta and theta power. Multivariate pattern classification analysis revealed that MW and meta-awareness can be decoded from the distribution of power in these three frequency bands. Critically, we show that individual decoding accuracies positively correlate with introspection confidence. Our results reaffirm the role of alpha oscillations in MW, implicate lower frequencies in meta-awareness, and are consistent with the proposal that introspection confidence indexes neurophysiological discriminability of representational states.
Collapse
Affiliation(s)
- Naya Polychroni
- Department of Psychology, Goldsmiths, University of London, London, UK
| | - Maria Herrojo Ruiz
- Department of Psychology, Goldsmiths, University of London, London, UK.,Center for Cognition and Decision Making, National Research University Higher School of Economics, Moscow, Russia
| | - Devin B Terhune
- Department of Psychology, Goldsmiths, University of London, London, UK
| |
Collapse
|
16
|
Hajonides JE, Nobre AC, van Ede F, Stokes MG. Decoding visual colour from scalp electroencephalography measurements. Neuroimage 2021; 237:118030. [PMID: 33836272 PMCID: PMC8285579 DOI: 10.1016/j.neuroimage.2021.118030] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2020] [Revised: 03/21/2021] [Accepted: 03/28/2021] [Indexed: 11/17/2022] Open
Abstract
Recent advances have made it possible to decode various aspects of visually presented stimuli from patterns of scalp EEG measurements. As of recently, such multivariate methods have been commonly used to decode visual-spatial features such as location, orientation, or spatial frequency. In the current study, we show that it is also possible to track visual colour processing by using Linear Discriminant Analysis on patterns of EEG activity. Building on other recent demonstrations, we show that colour decoding: (1) reflects sensory qualities (as opposed to, for example, verbal labelling) with a prominent contribution from posterior electrodes contralateral to the stimulus, (2) conforms to a parametric coding space, (3) is possible in multi-item displays, and (4) is comparable in magnitude to the decoding of visual stimulus orientation. Through subsampling our data, we also provide an estimate of the approximate number of trials and participants required for robust decoding. Finally, we show that while colour decoding can be sensitive to subtle differences in luminance, our colour decoding results are primarily driven by measured colour differences between stimuli. Colour decoding opens a relevant new dimension in which to track visual processing using scalp EEG measurements, while bypassing potential confounds associated with decoding approaches that focus on spatial features.
Collapse
Affiliation(s)
- Jasper E Hajonides
- Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, United Kingdom; Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom.
| | - Anna C Nobre
- Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, United Kingdom; Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Freek van Ede
- Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, United Kingdom; Institute for Brain and Behavior Amsterdam, Department of Experimental and Applied Psychology, Vrije Universiteit Amsterdam, Netherlands
| | - Mark G Stokes
- Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| |
Collapse
|
17
|
Varrier RS, Rothkirch M, Stuke H, Guggenmos M, Sterzer P. Unreliable feedback deteriorates information processing in primary visual cortex. Neuroimage 2020; 214:116701. [PMID: 32135261 DOI: 10.1016/j.neuroimage.2020.116701] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2019] [Revised: 02/01/2020] [Accepted: 02/29/2020] [Indexed: 11/24/2022] Open
Abstract
It is well-established that increased sensory uncertainty impairs perceptual decision-making and leads to degraded neural stimulus representations. Recently, we also showed that providing unreliable feedback to choices leads to changes in perceptual decision-making similar to those of increased stimulus noise: A deterioration in objective task performance, a decrease in subjective confidence and a lower reliance on sensory information for perceptual inference. To investigate the neural basis of such feedback-based changes in perceptual decision-making, in the present study, two groups of healthy human participants (n = 15 each) performed a challenging visual orientation discrimination task while undergoing functional magnetic resonance imaging (fMRI). Critically, one group received reliable feedback regarding their task performance in an intervention phase, whereas the other group correspondingly received unreliable feedback - thereby keeping stimulus information constant. The effects of feedback reliability on performance and stimulus representation in the primary visual cortex (V1) were studied by comparing the pre- and post-intervention test phases between the groups. Compared to participants who received reliable feedback, those receiving unreliable feedback showed a decline in task performance that was paralleled by reduced distinctness of fMRI response patterns in V1. These results show that environmental uncertainty can affect perceptual inference at the earliest cortical processing stages.
Collapse
Affiliation(s)
- Rekha S Varrier
- Department of Psychiatry and Psychotherapy, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and the Berlin Institute of Health, Chariteplatz 1, 10117, Berlin, Germany; Bernstein Center for Computational Neuroscience, Berlin, Humboldt University Berlin, 10115, Berlin, Germany.
| | - Marcus Rothkirch
- Department of Psychiatry and Psychotherapy, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and the Berlin Institute of Health, Chariteplatz 1, 10117, Berlin, Germany
| | - Heiner Stuke
- Department of Psychiatry and Psychotherapy, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and the Berlin Institute of Health, Chariteplatz 1, 10117, Berlin, Germany
| | - Matthias Guggenmos
- Department of Psychiatry and Psychotherapy, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and the Berlin Institute of Health, Chariteplatz 1, 10117, Berlin, Germany
| | - Philipp Sterzer
- Department of Psychiatry and Psychotherapy, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and the Berlin Institute of Health, Chariteplatz 1, 10117, Berlin, Germany; Bernstein Center for Computational Neuroscience, Berlin, Humboldt University Berlin, 10115, Berlin, Germany
| |
Collapse
|