1
|
Park J, Jeon JY, Kim R, Kay KN, Shim WM. Motion-corrected eye tracking (MoCET) improves gaze accuracy during visual fMRI experiments. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2025:2025.03.13.642919. [PMID: 40161851 PMCID: PMC11952553 DOI: 10.1101/2025.03.13.642919] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 04/02/2025]
Abstract
Human eye movements are essential for understanding cognition, yet achieving high-precision eye tracking in fMRI remains challenging. Even slight head shifts from the initial calibration position can introduce drift in eye tracking data, leading to substantial gaze inaccuracies. To address this, we introduce Motion-Corrected Eye Tracking (MoCET), a novel approach that corrects drift using head motion parameters derived from the preprocessing of fMRI data. MoCET requires no additional hardware and can be applied retrospectively to existing datasets. We show that it outperforms traditional detrending methods with respect to accuracy of gaze estimation and offers higher spatial and temporal precision compared to MR-based eye tracking approaches. By overcoming a key limitation in integrating eye tracking with fMRI, MoCET facilitates investigations of naturalistic vision and cognition in fMRI research.
Collapse
Affiliation(s)
- Jiwoong Park
- Center for Neuroscience Imaging Research, Institute for Basic Science (IBS), Republic of Korea
- Department of Biomedical Engineering, Sungkyunkwan University (SKKU), Republic of Korea
- Department of Intelligent Precision Healthcare Convergence, Sungkyunkwan University (SKKU), Republic of Korea
| | - Jae Young Jeon
- Center for Neuroscience Imaging Research, Institute for Basic Science (IBS), Republic of Korea
- Department of Intelligent Precision Healthcare Convergence, Sungkyunkwan University (SKKU), Republic of Korea
| | - Royoung Kim
- Center for Neuroscience Imaging Research, Institute for Basic Science (IBS), Republic of Korea
- Department of Intelligent Precision Healthcare Convergence, Sungkyunkwan University (SKKU), Republic of Korea
| | - Kendrick N. Kay
- Center for Magnetic Resonance Research, Department of Radiology, University of Minnesota
| | - Won Mok Shim
- Center for Neuroscience Imaging Research, Institute for Basic Science (IBS), Republic of Korea
- Department of Biomedical Engineering, Sungkyunkwan University (SKKU), Republic of Korea
- Department of Intelligent Precision Healthcare Convergence, Sungkyunkwan University (SKKU), Republic of Korea
| |
Collapse
|
2
|
Tang C, Gao T, Wang G, Chen B. Coherence-based channel selection and Riemannian geometry features for magnetoencephalography decoding. Cogn Neurodyn 2024; 18:3535-3548. [PMID: 39712116 PMCID: PMC11655792 DOI: 10.1007/s11571-024-10085-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Revised: 01/03/2024] [Accepted: 02/02/2024] [Indexed: 12/24/2024] Open
Abstract
Magnetoencephalography (MEG) records the extremely weak magnetic fields on the surface of the scalp through highly sensitive sensors. Multi-channel MEG data provide higher spatial and temporal resolution when measuring brain activities, and can be applied for brain-computer interfaces as well. However, a large number of channels leads to high computational complexity and can potentially impact decoding accuracy. To improve the accuracy of MEG decoding, this paper proposes a new coherence-based channel selection method that effectively identifies task-relevant channels, reducing the presence of noisy and redundant information. Riemannian geometry is then used to extract effective features from selected channels of MEG data. Finally, MEG decoding is achieved by training a support vector machine classifier with the Radial Basis Function kernel. Experiments were conducted on two public MEG datasets to validate the effectiveness of the proposed method. The results from Dataset 1 show that Riemannian geometry achieves higher classification accuracy (compared to common spatial patterns and power spectral density) in the single-subject visual decoding task. Moreover, coherence-based channel selection significantly (P = 0.0002) outperforms the use of all channels. Moving on to Dataset 2, the results reveal that coherence-based channel selection is also significantly (P <0.0001) superior to all channels and channels around C3 and C4 in cross-session mental imagery decoding tasks. Additionally, the proposed method outperforms state-of-the-art methods in motor imagery tasks.
Collapse
Affiliation(s)
- Chao Tang
- National Key Laboratory of Human–Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, 710049 China
| | - Tianyi Gao
- National Key Laboratory of Human–Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, 710049 China
| | - Gang Wang
- Key Laboratory of Biomedical Information Engineering of Ministry of Education, Xi’an Jiaotong University, Xi’an, 710049 China
| | - Badong Chen
- National Key Laboratory of Human–Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, 710049 China
| |
Collapse
|
3
|
Wang L, Zhou X, Zeng F, Cao M, Zuo S, Yang J, Kusunoki M, Wang H, Zhou YD, Chen A, Kwok SC. Mixed Selectivity Coding of Content-Temporal Detail by Dorsomedial Posterior Parietal Neurons. J Neurosci 2024; 44:e1677232023. [PMID: 37985178 PMCID: PMC10860630 DOI: 10.1523/jneurosci.1677-23.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Revised: 10/27/2023] [Accepted: 11/01/2023] [Indexed: 11/22/2023] Open
Abstract
The dorsomedial posterior parietal cortex (dmPPC) is part of a higher-cognition network implicated in elaborate processes underpinning memory formation, recollection, episode reconstruction, and temporal information processing. Neural coding for complex episodic processing is however under-documented. Here, we recorded extracellular neural activities from three male rhesus macaques (Macaca mulatta) and revealed a set of neural codes of "neuroethogram" in the primate parietal cortex. Analyzing neural responses in macaque dmPPC to naturalistic videos, we discovered several groups of neurons that are sensitive to different categories of ethogram items, low-level sensory features, and saccadic eye movement. We also discovered that the processing of category and feature information by these neurons is sustained by the accumulation of temporal information over a long timescale of up to 30 s, corroborating its reported long temporal receptive windows. We performed an additional behavioral experiment with additional two male rhesus macaques and found that saccade-related activities could not account for the mixed neuronal responses elicited by the video stimuli. We further observed monkeys' scan paths and gaze consistency are modulated by video content. Taken altogether, these neural findings explain how dmPPC weaves fabrics of ongoing experiences together in real time. The high dimensionality of neural representations should motivate us to shift the focus of attention from pure selectivity neurons to mixed selectivity neurons, especially in increasingly complex naturalistic task designs.
Collapse
Affiliation(s)
- Lei Wang
- Shanghai Key Laboratory of Brain Functional Genomics, Affiliated Mental Health Center (ECNU), School of Psychology and Cognitive Science, East China Normal University, Shanghai 200062, China
- Phylo-Cognition Laboratory, Division of Natural and Applied Sciences, Duke Kunshan University, Duke Institute for Brain Sciences, Kunshan 215316, Jiangsu, China
| | - Xufeng Zhou
- Shanghai Key Laboratory of Brain Functional Genomics, Affiliated Mental Health Center (ECNU), School of Psychology and Cognitive Science, East China Normal University, Shanghai 200062, China
- Phylo-Cognition Laboratory, Division of Natural and Applied Sciences, Duke Kunshan University, Duke Institute for Brain Sciences, Kunshan 215316, Jiangsu, China
| | - Fu Zeng
- Key Laboratory of Brain Functional Genomics (Ministry of Education), East China Normal University, Shanghai 200062, China
| | - Mingfeng Cao
- Phylo-Cognition Laboratory, Division of Natural and Applied Sciences, Duke Kunshan University, Duke Institute for Brain Sciences, Kunshan 215316, Jiangsu, China
- Whiting School of Engineering, department of biomedical engineering, Johns Hopkins University, Baltimore, Maryland 21218
| | - Shuzhen Zuo
- Shanghai Key Laboratory of Brain Functional Genomics, Affiliated Mental Health Center (ECNU), School of Psychology and Cognitive Science, East China Normal University, Shanghai 200062, China
- Laboratory for Circuit and Behavioral Physiology, RIKEN Center for Brain Science, Wako-shi, Saitama 351-0198, Japan
- Department of Life Sciences, Graduate School of Arts and Sciences, The University of Tokyo, Tokyo 153-8902, Japan
| | - Jie Yang
- Shanghai Key Laboratory of Brain Functional Genomics, Affiliated Mental Health Center (ECNU), School of Psychology and Cognitive Science, East China Normal University, Shanghai 200062, China
| | - Makoto Kusunoki
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge CB2 7EF, United Kingdom
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, United Kingdom
| | - Huimin Wang
- Shanghai Key Laboratory of Brain Functional Genomics, Affiliated Mental Health Center (ECNU), School of Psychology and Cognitive Science, East China Normal University, Shanghai 200062, China
- Key Laboratory of Brain Functional Genomics (Ministry of Education), East China Normal University, Shanghai 200062, China
- Shanghai Changning Mental Health Center, Shanghai 200335, China
- NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai, Shanghai 200062, China
| | - Yong-di Zhou
- School of Psychology, Shenzhen University, Shenzhen 518052, China
- Department of Neurosurgery, Johns Hopkins University, Baltimore, Maryland 21218
| | - Aihua Chen
- Key Laboratory of Brain Functional Genomics (Ministry of Education), East China Normal University, Shanghai 200062, China
| | - Sze Chai Kwok
- Shanghai Key Laboratory of Brain Functional Genomics, Affiliated Mental Health Center (ECNU), School of Psychology and Cognitive Science, East China Normal University, Shanghai 200062, China
- Phylo-Cognition Laboratory, Division of Natural and Applied Sciences, Duke Kunshan University, Duke Institute for Brain Sciences, Kunshan 215316, Jiangsu, China
- Key Laboratory of Brain Functional Genomics (Ministry of Education), East China Normal University, Shanghai 200062, China
- Shanghai Changning Mental Health Center, Shanghai 200335, China
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing 100875, China
- Shanghai Key Laboratory of Magnetic Resonance, East China Normal University, Shanghai 200062, China
| |
Collapse
|
4
|
Yang K, Hu Y, Zeng Y, Tong L, Gao Y, Pei C, Li Z, Yan B. EEG Network Analysis of Depressive Emotion Interference Spatial Cognition Based on a Simulated Robotic Arm Docking Task. Brain Sci 2023; 14:44. [PMID: 38248259 PMCID: PMC10813131 DOI: 10.3390/brainsci14010044] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Revised: 12/24/2023] [Accepted: 12/29/2023] [Indexed: 01/23/2024] Open
Abstract
Depressive emotion (DE) refers to clinically relevant depressive symptoms without meeting the diagnostic criteria for depression. Studies have demonstrated that DE can cause spatial cognition impairment. However, the brain network mechanisms underlying DE interference spatial cognition remain unclear. This study aimed to reveal the differences in brain network connections between DE and healthy control (HC) groups during resting state and a spatial cognition task. The longer operation time of the DE group during spatial cognition task indicated DE interference spatial cognition. In the resting state stage, the DE group had weaker network connections in theta and alpha bands than the HC group had. Specifically, the electrodes in parietal regions were hubs of the differential networks, which are related to spatial attention. Moreover, in docking task stages, the left frontoparietal network connections in delta, beta, and gamma bands were stronger in the DE group than those of the HC group. The enhanced left frontoparietal connections in the DE group may be related to brain resource reorganization to compensate for spatial cognition decline and ensure the completion of spatial cognition tasks. Thus, these findings might provide new insights into the neural mechanisms of depressive emotion interference spatial cognition.
Collapse
Affiliation(s)
- Kai Yang
- Henan Province Key Laboratory of Imaging and Intelligent Processing, PLA Strategy Support Force Information Engineering University, Zhengzhou 450001, China; (K.Y.)
| | - Yidong Hu
- Henan Province Key Laboratory of Imaging and Intelligent Processing, PLA Strategy Support Force Information Engineering University, Zhengzhou 450001, China; (K.Y.)
| | - Ying Zeng
- Henan Province Key Laboratory of Imaging and Intelligent Processing, PLA Strategy Support Force Information Engineering University, Zhengzhou 450001, China; (K.Y.)
- Key Laboratory for Neuroinformation of Ministry of Education, University of Electronic Science and Technology of China, Chengdu 611730, China
| | - Li Tong
- Henan Province Key Laboratory of Imaging and Intelligent Processing, PLA Strategy Support Force Information Engineering University, Zhengzhou 450001, China; (K.Y.)
| | - Yuanlong Gao
- Henan Province Key Laboratory of Imaging and Intelligent Processing, PLA Strategy Support Force Information Engineering University, Zhengzhou 450001, China; (K.Y.)
| | - Changfu Pei
- Henan Province Key Laboratory of Imaging and Intelligent Processing, PLA Strategy Support Force Information Engineering University, Zhengzhou 450001, China; (K.Y.)
| | - Zhongrui Li
- Henan Province Key Laboratory of Imaging and Intelligent Processing, PLA Strategy Support Force Information Engineering University, Zhengzhou 450001, China; (K.Y.)
| | - Bin Yan
- Henan Province Key Laboratory of Imaging and Intelligent Processing, PLA Strategy Support Force Information Engineering University, Zhengzhou 450001, China; (K.Y.)
| |
Collapse
|
5
|
Zhang B, Naya Y. A dataset of human fMRI/MEG experiments with eye tracking for spatial memory research using virtual reality. Data Brief 2022; 43:108380. [PMID: 35789905 PMCID: PMC9249601 DOI: 10.1016/j.dib.2022.108380] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Revised: 06/02/2022] [Accepted: 06/07/2022] [Indexed: 11/30/2022] Open
Abstract
A dataset consisting of whole-brain fMRI (functional magnetic resonance imaging)/MEG (magnetoencephalography) images, eye tracking files, and behavioral records from healthy adult human participants when they performed a spatial-memory paradigm in a virtual environment was collected to investigate the neural representation of the cognitive map defined by unique spatial relationship of three objects, as well as the neural dynamics of the cognitive map following the task demand from localizing self-location to remembering the target location relative to the self-body. The dataset, including both fMRI and MEG, was also used to investigate the neural networks involved in representing a target within and outside the visual field. The dataset included 19 and 12 university students at Peking University for fMRI and MEG experiments, respectively (fMRI: 12 women, 7 men; MEG: 4 women, 8 men). The average ages of those participants were 24.9 years (MRI: 18–30 years) and 22.5 years (MEG: 19–25 years), respectively. fMRI BOLD and T1-weighted images were acquired using a 3T Siemens Prisma scanner (Siemens, Erlangen, Germany) equipped with a 20-channel receiver head coil. MEG neuromagnetic data were acquired using a 275-channel MEG system (CTF MEG, Canada). The dataset could be further used to investigate a range of neural mechanisms involved in human spatial cognition or to develop a bioinspired deep neural network to enhance machines' abilities in spatial processing.
Collapse
Affiliation(s)
- Bo Zhang
- Beijing Academy of Artificial Intelligence, Beijing 100084, China
- Tsinghua Laboratory of Brain and Intelligence, Tsinghua University, 160 Chengfu Rd., SanCaiTang Building, Haidian District, Beijing 100084, China
- Corresponding author at: Beijing Academy of Artificial Intelligence, Beijing 100084, China.
| | - Yuji Naya
- School of Psychological and Cognitive Sciences, Peking University, No. 52, Haidian Road, Haidian District, Beijing 100805, China
- IDG/McGovern Institute for Brain Research at Peking University, No. 52, Haidian Road, Haidian District, Beijing 100805, China
- Center for Life Sciences, Peking University, No. 52, Haidian Road, Haidian District, Beijing 100805, China
- Beijing Key Laboratory of Behavior and Mental Health, Peking University, No. 52, Haidian Road, Haidian District, Beijing 100805, China
- Corresponding author at: School of Psychological and Cognitive Sciences, Peking University, No. 52, Haidian Road, Haidian District, Beijing 100805, China.
| |
Collapse
|