Pattern classification precedes region-average hemodynamic response in early visual cortex

Imaging faster neural dynamics with fast fMRI: A need for updated models of the hemodynamic response

Fast fMRI enables the detection of neural dynamics over timescales of hundreds of milliseconds, suggesting it may provide a new avenue for studying subsecond neural processes in the human brain. The magnitudes of these fast fMRI dynamics are far greater than predicted by canonical models of the hemodynamic response. Several studies have established nonlinear properties of the hemodynamic response that have significant implications for fast fMRI. We first review nonlinear properties of the hemodynamic response function that may underlie fast fMRI signals. We then illustrate the breakdown of canonical hemodynamic response models in the context of fast neural dynamics. We will then argue that the canonical hemodynamic response function is not likely to reflect the BOLD response to neuronal activity driven by sparse or naturalistic stimuli or perhaps to spontaneous neuronal fluctuations in the resting state. These properties suggest that fast fMRI is capable of tracking surprisingly fast neuronal dynamics, and we discuss the neuroscientific questions that could be addressed using this approach.

Decoding Multiple Sound-Categories in the Auditory Cortex by Neural Networks: An fNIRS Study

This study aims to decode the hemodynamic responses (HRs) evoked by multiple sound-categories using functional near-infrared spectroscopy (fNIRS). The six different sounds were given as stimuli (English, non-English, annoying, nature, music, and gunshot). The oxy-hemoglobin (HbO) concentration changes are measured in both hemispheres of the auditory cortex while 18 healthy subjects listen to 10-s blocks of six sound-categories. Long short-term memory (LSTM) networks were used as a classifier. The classification accuracy was 20.38 ± 4.63% with six class classification. Though LSTM networks' performance was a little higher than chance levels, it is noteworthy that we could classify the data subject-wise without feature selections.

Deep-Learning-Based Multivariate Pattern Analysis (dMVPA): A Tutorial and a Toolbox

In recent years, multivariate pattern analysis (MVPA) has been hugely beneficial for cognitive neuroscience by making new experiment designs possible and by increasing the inferential power of functional magnetic resonance imaging (fMRI), electroencephalography (EEG), and other neuroimaging methodologies. In a similar time frame, “deep learning” (a term for the use of artificial neural networks with convolutional, recurrent, or similarly sophisticated architectures) has produced a parallel revolution in the field of machine learning and has been employed across a wide variety of applications. Traditional MVPA also uses a form of machine learning, but most commonly with much simpler techniques based on linear calculations; a number of studies have applied deep learning techniques to neuroimaging data, but we believe that those have barely scratched the surface of the potential deep learning holds for the field. In this paper, we provide a brief introduction to deep learning for those new to the technique, explore the logistical pros and cons of using deep learning to analyze neuroimaging data – which we term “deep MVPA,” or dMVPA – and introduce a new software toolbox (the “Deep Learning In Neuroimaging: Exploration, Analysis, Tools, and Education” package, DeLINEATE for short) intended to facilitate dMVPA for neuroscientists (and indeed, scientists more broadly) everywhere.

Temporal embedding and spatiotemporal feature selection boost multi-voxel pattern analysis decoding accuracy

BACKGROUND

In fMRI decoding, temporal embedding of spatial features of the brain allows the incorporation of brain activity dynamics into the multivariate pattern classification process, and provides enriched information about stimulus-specific response patterns and potentially improved prediction accuracy.

NEW METHOD

This study investigates the possibility of enhancing the classification performance by exploring temporal embedding, to identify the optimum combination of spatiotemporal features based on their classification performance. We investigated the importance of spatiotemporal feature selection using a slow event-related design adapted from the classic Haxby study (Haxby et al., 2001). Data were collected using a multiband fMRI sequence with temporal resolution of 0.568 s.

COMPARISON WITH EXISTING METHODS

A wide range of spatiotemporal observations were created as various combinations of spatiotemporal features. Using both random forest, and support vector machine, classifiers prediction accuracies for these combinations were then compared with the single spatial multivariate pattern approach that uses only a single temporal observation.

RESULTS

Our findings showed that, on average, spatiotemporal feature selection improved prediction accuracy. Moreover, the random forest algorithm outperformed the support vector machine and benefitted from temporal information to a greater extent.

CONCLUSIONS

As expected, the most influential temporal durations were found to be around the peak of the hemodynamic response function, a few seconds after the stimuli onset until -4 s after the peak of the hemodynamic response function. The superiority of spatiotemporal feature selection over single time-point spatial approaches invites future work to design optimal approaches that incorporate spatiotemporal dependencies into feature selection for decoding.

Dissociated modulations of multivoxel activation patterns in the ventral and dorsal visual pathways by the temporal dynamics of stimuli

INTRODUCTION

Previous studies suggested temporal limitations of visual object identification in the ventral pathway. Moreover, multivoxel pattern analyses (MVPA) of fMRI activation have shown reliable encoding of various object categories including faces and tools in the ventral pathway. By contrast, the dorsal pathway is involved in reaching a target and grasping a tool, and quicker in processing the temporal dynamics of stimulus change. However, little is known about how activation patterns in both pathways may change according to the temporal dynamics of stimulus change.

METHODS

Here, we measured fMRI responses of two consecutive stimuli with varying interstimulus intervals (ISIs), and we compared how the two visual pathways respond to the dynamics of stimuli by using MVPA and information-based searchlight mapping.

RESULTS

We found that the temporal dynamics of stimuli modulate responses of the two visual pathways in opposite directions. Specifically, slower temporal dynamics (longer ISIs) led to greater activity and better MVPA results in the ventral pathway. However, faster temporal dynamics (shorter ISIs) led to greater activity and better MVPA results in the dorsal pathway.

CONCLUSIONS

These results are the first to show how temporal dynamics of stimulus change modulated multivoxel fMRI activation pattern change. And such temporal dynamic response function in different ROIs along the two visual pathways may shed lights on understanding functional relationship and organization of these ROIs.

Seeing patterns in neuroimaging data

Functional magnetic resonance imaging research is often associated with images of brains overlaid with patterns of color that indicate significant activity. These images are one of the most salient and recognizable pieces of evidence neuroscientists appeal to as justification for claims about the relationship between cognitive processes and human behavior. The strongest critics of neuroimaging research argue that the technology possesses little, if any, scientific value, in part because of the assumptions implicit in the complex analysis procedures used to transform the data into interpretable data patterns. In this chapter, I shift the focus of this debate away from assumptions implicit in the operation of techniques themselves, and toward the role data analysis techniques play as parts of the process of interpreting neuroimaging data. I propose that data analysis techniques can be conceived of as a lens that brings patterns within the data into focus through its selective transformation. This approach recognizes the double-edged nature of data analysis and interpretation: techniques render data interpretable, but their selection and application is often informed by the methodological and theoretical commitments of researchers using them.

Reproducibility of importance extraction methods in neural network based fMRI classification

Recent advances in machine learning allow faster training, improved performance and increased interpretability of classification techniques. Consequently, their application in neuroscience is rapidly increasing. While classification approaches have proved useful in functional magnetic resonance imaging (fMRI) studies, there are concerns regarding extraction, reproducibility and visualization of brain regions that contribute most significantly to the classification. We addressed these issues using an fMRI classification scheme based on neural networks and compared a set of methods for extraction of category-related voxel importances in three simulated and two empirical datasets. The simulation data revealed that the proposed scheme successfully detects spatially distributed and overlapping activation patterns upon successful classification. Application of the proposed classification scheme to two previously published empirical fMRI datasets revealed robust importance maps that extensively overlap with univariate maps but also provide complementary information. Our results demonstrate increased statistical power of importance maps compared to univariate approaches for both detection of overlapping patterns and patterns with weak univariate information.

Neural predictors of evaluative attitudes toward celebrities

Our attitudes toward others influence a wide range of everyday behaviors and have been the most extensively studied concept in the history of social psychology. Yet they remain difficult to measure reliably and objectively, since both explicit and implicit measures are typically confounded by other psychological processes. We here address the feasibility of decoding incidental attitudes based on brain activations. Participants were presented with pictures of members of a Japanese idol group inside an functional magnetic resonance imaging scanner while performing an unrelated detection task, and subsequently (outside the scanner) performed an incentive-compatible choice task that revealed their attitude toward each celebrity. We used a real-world election scheme that exists for this idol group, which confirmed both strongly negative and strongly positive attitudes toward specific individuals. Whole-brain multivariate analyses (searchlight-based support vector regression) showed that activation patterns in the anterior striatum predicted each participant’s revealed attitudes (choice behavior) using leave-one-out (as well as 4-fold) cross-validation across participants. In contrast, attitude extremity (unsigned magnitude) could be decoded from a distinct region in the posterior striatum. The findings demonstrate dissociable striatal representations of valenced attitude and attitude extremity and constitute a first step toward an objective and process-pure neural measure of attitudes.

From connectome to cognition: The search for mechanism in human functional brain networks

Recent developments in functional connectivity research have expanded the scope of human neuroimaging, from identifying changes in regional activation amplitudes to detailed mapping of large-scale brain networks. However, linking network processes to a clear role in cognition demands advances in the theoretical frameworks, algorithms, and experimental approaches applied. This would help evolve the field from a descriptive to an explanatory state, by targeting network interactions that can mechanistically account for cognitive effects. In the present review, we provide an explicit framework to aid this search for "network mechanisms", which anchors recent methodological advances in functional connectivity estimation to a renewed emphasis on careful experimental design. We emphasize how this framework can address specific questions in network neuroscience. These span ambiguity over the cognitive relevance of resting-state networks, how to characterize task-evoked and spontaneous network dynamics, how to identify directed or "effective" connections, and how to apply multivariate pattern analysis at the network level. In parallel, we apply the framework to highlight the mechanistic interaction of network components that remain "stable" across task domains and more "flexible" components associated with on-task reconfiguration. By emphasizing the need to structure the use of diverse analytic approaches with sound experimentation, our framework promotes an explanatory mapping between the workings of the cognitive mind and the large-scale network mechanisms of the human brain.

Fundamentally Distributed Information Processing Integrates the Motor Network into the Mental Workspace during Mental Rotation

The manipulation of mental representations in the human brain appears to share similarities with the physical manipulation of real-world objects. In particular, some neuroimaging studies have found increased activity in motor regions during mental rotation, suggesting that mental and physical operations may involve overlapping neural populations. Does the motor network contribute information processing to mental rotation? If so, does it play a similar computational role in both mental and manual rotation, and how does it communicate with the wider network of areas involved in the mental workspace? Here we used multivariate methods and fMRI to study 24 participants as they mentally rotated 3-D objects or manually rotated their hands in one of four directions. We find that information processing related to mental rotations is distributed widely among many cortical and subcortical regions, that the motor network becomes tightly integrated into a wider mental workspace network during mental rotation, and that motor network activity during mental rotation only partially resembles that involved in manual rotation. Additionally, these findings provide evidence that the mental workspace is organized as a distributed core network that dynamically recruits specialized subnetworks for specific tasks as needed.

Decoding neural representational spaces using multivariate pattern analysis

A major challenge for systems neuroscience is to break the neural code. Computational algorithms for encoding information into neural activity and extracting information from measured activity afford understanding of how percepts, memories, thought, and knowledge are represented in patterns of brain activity. The past decade and a half has seen significant advances in the development of methods for decoding human neural activity, such as multivariate pattern classification, representational similarity analysis, hyperalignment, and stimulus-model-based encoding and decoding. This article reviews these advances and integrates neural decoding methods into a common framework organized around the concept of high-dimensional representational spaces.