Accurately decoding visual information from fMRI data obtained in a realistic virtual environment

Representations within the Intraparietal Sulcus Distinguish Numerical Tasks and Formats

How does our brain understand the number five when it is written as an Arabic numeral, and when presented as five fingers held up? Four facets have been implicated in adult numerical processing: semantic, visual, manual, and phonological/verbal. Here, we ask how the brain represents each, using a combination of tasks and stimuli. We collected fMRI data from adult participants while they completed our novel "four number code" paradigm. In this paradigm, participants viewed one of two stimulus types to tap into the visual and manual number codes, respectively. Concurrently, they completed one of two tasks to tap into the semantic and phonological/verbal number codes, respectively. Classification analyses revealed that neural codes representing distinctions between the number comparison and phonological tasks were generalizable across format (e.g., Arabic numerals to hands) within intraparietal sulcus (IPS), angular gyrus, and precentral gyrus. Neural codes representing distinctions between formats were generalizable across tasks within visual areas such as fusiform gyrus and calcarine sulcus, as well as within IPS. Our results identify the neural facets of numerical processing within a single paradigm and suggest that IPS is sensitive to distinctions between semantic and phonological/verbal, as well as visual and manual, facets of number representations.

Framework for Accurate Classification of Self-Reported Stress From Multisession Functional MRI Data of Veterans With Posttraumatic Stress

Background: Posttraumatic stress disorder (PTSD) is a significant burden among combat Veterans returning from the wars in Iraq and Afghanistan. While empirically supported treatments have demonstrated reductions in PTSD symptomatology, there remains a need to improve treatment effectiveness. Functional magnetic resonance imaging (fMRI) neurofeedback has emerged as a possible treatment to ameliorate PTSD symptom severity. Virtual reality (VR) approaches have also shown promise in increasing treatment compliance and outcomes. To facilitate fMRI neurofeedback-associated therapies, it would be advantageous to accurately classify internal brain stress levels while Veterans are exposed to trauma-associated VR imagery. Methods: Across 2 sessions, we used fMRI to collect neural responses to trauma-associated VR-like stimuli among male combat Veterans with PTSD symptoms (N = 8). Veterans reported their self-perceived stress level on a scale from 1 to 8 every 15 s throughout the fMRI sessions. In our proposed framework, we precisely sample the fMRI data on cortical gray matter, blurring the data along the gray-matter manifold to reduce noise and dimensionality while preserving maximum neural information. Then, we independently applied 3 machine learning (ML) algorithms to this fMRI data collected across 2 sessions, separately for each Veteran, to build individualized ML models that predicted their internal brain states (self-reported stress responses). Results: We accurately classified the 8-class self-reported stress responses with a mean (± standard error) root mean square error of 0.6 (± 0.1) across all Veterans using the best ML approach. Conclusions: The findings demonstrate the predictive ability of ML algorithms applied to whole-brain cortical fMRI data collected during individual Veteran sessions. The framework we have developed to preprocess whole-brain cortical fMRI data and train ML models across sessions would provide a valuable tool to enable individualized real-time fMRI neurofeedback during VR-like exposure therapy for PTSD.

Decoding task specific and task general functional architectures of the brain

Functional magnetic resonance imaging (fMRI) is used to capture complex and dynamic interactions between brain regions while performing tasks. Task related alterations in the brain have been classified as task specific and task general, depending on whether they are particular to a task or common across multiple tasks. Using recent attempts in interpreting deep learning models, we propose an approach to determine both task specific and task general architectures of the functional brain. We demonstrate our methods with a reference‐based decoder on deep learning classifiers trained on 12,500 rest and task fMRI samples from the Human Connectome Project (HCP). The decoded task general and task specific motor and language architectures were validated with findings from previous studies. We found that unlike intersubject variability that is characteristic of functional pathology of neurological diseases, a small set of connections are sufficient to delineate the rest and task states. The nodes and connections in the task general architecture could serve as potential disease biomarkers as alterations in task general brain modulations are known to be implicated in several neuropsychiatric disorders.

Experimental and methodological factors affecting test-retest reliability of amygdala BOLD responses

Previous studies reported poor to fair test-retest reliability of amygdala BOLD responses to emotional stimuli. However, these findings are very heterogeneous across and within studies. The present study sought to systematically examine experimental and methodological factors that contribute to this heterogeneity. Forty-six young subjects were scanned twice with a mean test-retest interval of 7 weeks. We compared amygdala reliability across three tasks: A face-matching task, passive viewing of emotional faces, and passive viewing of emotional scenes. We also explored whether extraction of physiological noise can affect the stability of amygdala responses. We assessed test-retest reliability of amygdala mean amplitudes at the individual level and spatial repeatability (i.e., stability of the spatial distribution of activation) of the amygdala BOLD signal at the group and individual level. All three tasks evoked robust amygdala activation at the group level. At the individual level, amygdala spatial repeatability was poor during passive viewing of scenes and faces and fair or close to fair in the face-matching task. On the other hand, reliability of amygdala mean responses was very poor in the face-matching task while it was significantly higher during passive viewing of faces and scenes. Physiological noise correction changed reliability rates but not uniformly across the three tasks. The current work suggests that the presence of a concurrent task during emotion processing affects amygdala reliability. The dissociation between spatial repeatability and reliability of mean amplitudes highlights the importance of taking into account both measures for a multidimensional assessment of the reliability of BOLD responses.

Reproducibility of importance extraction methods in neural network based fMRI classification

Recent advances in machine learning allow faster training, improved performance and increased interpretability of classification techniques. Consequently, their application in neuroscience is rapidly increasing. While classification approaches have proved useful in functional magnetic resonance imaging (fMRI) studies, there are concerns regarding extraction, reproducibility and visualization of brain regions that contribute most significantly to the classification. We addressed these issues using an fMRI classification scheme based on neural networks and compared a set of methods for extraction of category-related voxel importances in three simulated and two empirical datasets. The simulation data revealed that the proposed scheme successfully detects spatially distributed and overlapping activation patterns upon successful classification. Application of the proposed classification scheme to two previously published empirical fMRI datasets revealed robust importance maps that extensively overlap with univariate maps but also provide complementary information. Our results demonstrate increased statistical power of importance maps compared to univariate approaches for both detection of overlapping patterns and patterns with weak univariate information.

Listening in Naturalistic Scenes: What Can Functional Near-Infrared Spectroscopy and Intersubject Correlation Analysis Tell Us About the Underlying Brain Activity?

Listening to speech in the noisy conditions of everyday life can be effortful, reflecting the increased cognitive workload involved in extracting meaning from a degraded acoustic signal. Studying the underlying neural processes has the potential to provide mechanistic insight into why listening is effortful under certain conditions. In a move toward studying listening effort under ecologically relevant conditions, we used the silent and flexible neuroimaging technique functional near-infrared spectroscopy (fNIRS) to examine brain activity during attentive listening to speech in naturalistic scenes. Thirty normally hearing participants listened to a series of narratives continuously varying in acoustic difficulty while undergoing fNIRS imaging. Participants then listened to another set of closely matched narratives and rated perceived effort and intelligibility for each scene. As expected, self-reported effort generally increased with worsening signal-to-noise ratio. After controlling for better-ear signal-to-noise ratio, perceived effort was greater in scenes that contained competing speech than in those that did not, potentially reflecting an additional cognitive cost of overcoming informational masking. We analyzed the fNIRS data using intersubject correlation, a data-driven approach suitable for analyzing data collected under naturalistic conditions. Significant intersubject correlation was seen in the bilateral auditory cortices and in a range of channels across the prefrontal cortex. The involvement of prefrontal regions is consistent with the notion that higher order cognitive processes are engaged during attentive listening to speech in complex real-world conditions. However, further research is needed to elucidate the relationship between perceived listening effort and activity in these extended cortical networks.