1
|
da Silva Soares R, Ramirez-Chavez KL, Tufanoglu A, Barreto C, Sato JR, Ayaz H. Cognitive Effort during Visuospatial Problem Solving in Physical Real World, on Computer Screen, and in Virtual Reality. SENSORS (BASEL, SWITZERLAND) 2024; 24:977. [PMID: 38339693 PMCID: PMC10857420 DOI: 10.3390/s24030977] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/13/2023] [Revised: 01/30/2024] [Accepted: 01/30/2024] [Indexed: 02/12/2024]
Abstract
Spatial cognition plays a crucial role in academic achievement, particularly in science, technology, engineering, and mathematics (STEM) domains. Immersive virtual environments (VRs) have the growing potential to reduce cognitive load and improve spatial reasoning. However, traditional methods struggle to assess the mental effort required for visuospatial processes due to the difficulty in verbalizing actions and other limitations in self-reported evaluations. In this neuroergonomics study, we aimed to capture the neural activity associated with cognitive workload during visuospatial tasks and evaluate the impact of the visualization medium on visuospatial task performance. We utilized functional near-infrared spectroscopy (fNIRS) wearable neuroimaging to assess cognitive effort during spatial-reasoning-based problem-solving and compared a VR, a computer screen, and a physical real-world task presentation. Our results reveal a higher neural efficiency in the prefrontal cortex (PFC) during 3D geometry puzzles in VR settings compared to the settings in the physical world and on the computer screen. VR appears to reduce the visuospatial task load by facilitating spatial visualization and providing visual cues. This makes it a valuable tool for spatial cognition training, especially for beginners. Additionally, our multimodal approach allows for progressively increasing task complexity, maintaining a challenge throughout training. This study underscores the potential of VR in developing spatial skills and highlights the value of comparing brain data and human interaction across different training settings.
Collapse
Affiliation(s)
- Raimundo da Silva Soares
- School of Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA 19104, USA; (K.L.R.-C.); (A.T.); (C.B.)
- Center of Mathematics Computation and Cognition, Universidade Federal do ABC, São Bernardo do Campo 09606-405, Brazil;
| | - Kevin L. Ramirez-Chavez
- School of Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA 19104, USA; (K.L.R.-C.); (A.T.); (C.B.)
| | - Altona Tufanoglu
- School of Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA 19104, USA; (K.L.R.-C.); (A.T.); (C.B.)
| | - Candida Barreto
- School of Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA 19104, USA; (K.L.R.-C.); (A.T.); (C.B.)
| | - João Ricardo Sato
- Center of Mathematics Computation and Cognition, Universidade Federal do ABC, São Bernardo do Campo 09606-405, Brazil;
| | - Hasan Ayaz
- School of Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA 19104, USA; (K.L.R.-C.); (A.T.); (C.B.)
- Department of Psychological and Brain Sciences, College of Arts and Sciences, Drexel University, Philadelphia, PA 19104, USA
- Drexel Solutions Institute, Drexel University, Philadelphia, PA 19104, USA
- A.J. Drexel Autism Institute, Drexel University, Philadelphia, PA 19104, USA
- Department of Family and Community Health, University of Pennsylvania, Philadelphia, PA 19104, USA
- Center for Injury Research and Prevention, Children’s Hospital of Philadelphia, Philadelphia, PA 19104, USA
| |
Collapse
|
2
|
da Silva Soares R, Ambriola Oku AY, Barreto CSF, Ricardo Sato J. Applying functional near-infrared spectroscopy and eye-tracking in a naturalistic educational environment to investigate physiological aspects that underlie the cognitive effort of children during mental rotation tests. Front Hum Neurosci 2022; 16:889806. [PMID: 36072886 PMCID: PMC9442578 DOI: 10.3389/fnhum.2022.889806] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2022] [Accepted: 06/27/2022] [Indexed: 11/24/2022] Open
Abstract
Spatial cognition is related to academic achievement in science, technology, engineering, and mathematics (STEM) domains. Neuroimaging studies suggest that brain regions' activation might be related to the general cognitive effort while solving mental rotation tasks (MRT). In this study, we evaluate the mental effort of children performing MRT tasks by measuring brain activation and pupil dilation. We use functional near-infrared spectroscopy (fNIRS) concurrently to collect brain hemodynamic responses from children's prefrontal cortex (PFC) and an Eye-tracking system to measure pupil dilation during MRT. Thirty-two healthy students aged 9-11 participated in this experiment. Behavioral measurements such as task performance on geometry problem-solving tests and MRT scores were also collected. The results were significant positive correlations between the children's MRT and geometry problem-solving test scores. There are also significant positive correlations between dorsolateral PFC (dlPFC) hemodynamic signals and visuospatial task performances (MRT and geometry problem-solving scores). Moreover, we found significant activation in the amplitude of deoxy-Hb variation on the dlPFC and that pupil diameter increased during the MRT, suggesting that both physiological responses are related to mental effort processes during the visuospatial task. Our findings indicate that children with more mental effort under the task performed better. The multimodal approach to monitoring students' mental effort can be of great interest in providing objective feedback on cognitive resource conditions and advancing our comprehension of the neural mechanisms that underlie cognitive effort. Hence, the ability to detect two distinct mental states of rest or activation of children during the MRT could eventually lead to an application for investigating the visuospatial skills of young students using naturalistic educational paradigms.
Collapse
Affiliation(s)
- Raimundo da Silva Soares
- Center for Mathematics, Computation and Cognition, Universidade Federal do ABC, São Bernardo do Campo, Brazil
- Graduate Program in Neuroscience and Cognition, Federal University of ABC, São Bernardo do Campo, Brazil
| | - Amanda Yumi Ambriola Oku
- Center for Mathematics, Computation and Cognition, Universidade Federal do ABC, São Bernardo do Campo, Brazil
| | - Cândida S. F. Barreto
- South African National Research Foundation Research Chair, Faculty of Education, University of Johannesburg, Johannesburg, South Africa
| | - João Ricardo Sato
- Center for Mathematics, Computation and Cognition, Universidade Federal do ABC, São Bernardo do Campo, Brazil
| |
Collapse
|
3
|
Weyand S, Takehara-Nishiuchi K, Chau T. Weaning Off Mental Tasks to Achieve Voluntary Self-Regulatory Control of a Near-Infrared Spectroscopy Brain-Computer Interface. IEEE Trans Neural Syst Rehabil Eng 2015; 23:548-61. [DOI: 10.1109/tnsre.2015.2399392] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
4
|
Weyand S, Schudlo L, Takehara-Nishiuchi K, Chau T. Usability and performance-informed selection of personalized mental tasks for an online near-infrared spectroscopy brain-computer interface. NEUROPHOTONICS 2015; 2:025001. [PMID: 26158005 PMCID: PMC4478988 DOI: 10.1117/1.nph.2.2.025001] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/13/2015] [Accepted: 04/10/2015] [Indexed: 05/29/2023]
Abstract
Brain-computer interfaces (BCIs) allow individuals to use only cognitive activities to interact with their environment. The widespread use of BCIs is limited, due in part to their lack of user-friendliness. The main goal of this work was to develop a more user-centered BCI and determine if: (1) individuals can acquire control of an online near-infrared spectroscopy BCI via usability and performance-informed selection of mental tasks without compromising classification accuracy and (2) the combination of usability and performance-informed selection of mental tasks yields subjective ease-of-use ratings that exceed those attainable with prescribed mental tasks. Twenty able-bodied participants were recruited. Half of the participants served as a control group, using the state-of-the-art prescribed mental strategies. The other half of the participants comprised the study group, choosing their own personalized mental strategies out of eleven possible tasks. It was concluded that users were, in fact, able to acquire control of the more user-centered BCI without a significant change in accuracy compared to the prescribed task BCI. Furthermore, the personalized BCI yielded higher subjective ease-of-use ratings than the prescribed BCI. Average online accuracies of [Formula: see text] and [Formula: see text] were achieved by the personalized and prescribed mental task groups, respectively.
Collapse
Affiliation(s)
- Sabine Weyand
- Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, Ontario M4G 1R8, Canada
- University of Toronto, Institute of Biomaterials and Biomedical Engineering, Ontario M5S 3G9, Canada
| | - Larissa Schudlo
- Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, Ontario M4G 1R8, Canada
- University of Toronto, Institute of Biomaterials and Biomedical Engineering, Ontario M5S 3G9, Canada
| | | | - Tom Chau
- Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, Ontario M4G 1R8, Canada
- University of Toronto, Institute of Biomaterials and Biomedical Engineering, Ontario M5S 3G9, Canada
| |
Collapse
|
5
|
Gentili RJ, Shewokis PA, Ayaz H, Contreras-Vidal JL. Functional near-infrared spectroscopy-based correlates of prefrontal cortical dynamics during a cognitive-motor executive adaptation task. Front Hum Neurosci 2013; 7:277. [PMID: 23847489 PMCID: PMC3701150 DOI: 10.3389/fnhum.2013.00277] [Citation(s) in RCA: 41] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2012] [Accepted: 05/27/2013] [Indexed: 11/13/2022] Open
Abstract
This study investigated changes in brain hemodynamics, as measured by functional near infrared spectroscopy, during performance of a cognitive-motor adaptation task. The adaptation task involved the learning of a novel visuomotor transformation (a 60° counterclockwise screen-cursor rotation), which required inhibition of a prepotent visuomotor response. A control group experienced a familiar transformation and thus, did not face any executive challenge. Analysis of the experimental group hemodynamic responses revealed that the performance enhancement was associated with a monotonic reduction in the oxygenation level in the prefrontal cortex. This finding confirms and extends functional magnetic resonance imaging and electroencephalography studies of visuomotor adaptation and learning. The changes in prefrontal brain activation suggest an initial recruitment of frontal executive functioning to inhibit prepotent visuomotor mappings followed by a progressive de-recruitment of the same prefrontal regions. The prefrontal hemodynamic changes observed in the experimental group translated into enhanced motor performance revealed by a reduction in movement time, movement extent, root mean square error and the directional error. These kinematic adaptations are consistent with the acquisition of an internal model of the novel visuomotor transformation. No comparable change was observed in the control group for either the hemodynamics or for the kinematics. This study (1) extends our understanding of the frontal executive processes from the cognitive to the cognitive-motor domain and (2) suggests that optical brain imaging can be employed to provide hemodynamic based-biomarkers to assess and monitor the level of adaptive cognitive-motor performance.
Collapse
Affiliation(s)
- Rodolphe J Gentili
- Department of Kinesiology, School of Public Health, University of Maryland College Park, MD, USA ; Graduate Program in Neuroscience and Cognitive Science, University of Maryland College Park, MD, USA ; Maryland Robotics Center, University of Maryland College Park, MD, USA
| | | | | | | |
Collapse
|