The effect of acquisition resolution on orientation decoding from V1 BOLD fMRI at 7T

Discriminating orientation information with phase consistency in alpha and low-gamma frequency bands: an EEG study

Recent studies suggest that noninvasive imaging methods (EEG, MEG) in the human brain scalp can decode the content of visual features information (orientation, color, motion, etc.) in Visual-Working Memory (VWM). Previous work demonstrated that with the sustained low-frequency Event-Related Potential (ERP under 6 Hz) of scalp EEG distributions, it is possible to accurately decode the content of orientation information in VWM during the delay interval. In addition, previous studies showed that the raw data captured by a combination of the occi-parietal electrodes could be used to decode the orientation. However, it is unclear whether the orientation information is available in other frequency bands (higher than 6 Hz) or whether this information is feasible with fewer electrodes. Furthermore, the exploration of orientation information in the phase values of the signal has not been well-addressed. In this study, we propose that orientation information is also accessible through the phase consistency of the occipital region in the alpha band frequency. Our results reveal a significant difference between orientations within 200 ms after stimulus offset in early visual sensory processing, with no apparent effect in power and Event-Related Oscillation (ERO) during this period. Additionally, in later periods (420-500 ms after stimulus offset), a noticeable difference is observed in the phase consistency of low gamma-band activity in the occipital area. Importantly, our findings suggest that phase consistency between trials of the orientation feature in the occipital alpha and low gamma-band can serve as a measure to obtain orientation information in VWM. Furthermore, the study demonstrates that phase consistency in the alpha and low gamma band can reflect the distribution of orientation-selective neuron numbers in the four main orientations in the occipital area.

Representational Geometries Reveal Differential Effects of Response Correlations on Population Codes in Neurophysiology and Functional Magnetic Resonance Imaging

Two sensory neurons usually display trial-by-trial spike-count correlations given the repeated representations of a stimulus. The effects of such response correlations on population-level sensory coding have been the focal contention in computational neuroscience over the past few years. In the meantime, multivariate pattern analysis (MVPA) has become the leading analysis approach in functional magnetic resonance imaging (fMRI), but the effects of response correlations among voxel populations remain underexplored. Here, instead of conventional MVPA analysis, we calculate linear Fisher information of population responses in human visual cortex (five males, one female) and hypothetically remove response correlations between voxels. We found that voxelwise response correlations generally enhance stimulus information, a result standing in stark contrast to the detrimental effects of response correlations reported in empirical neurophysiological studies. By voxel-encoding modeling, we further show that these two seemingly opposite effects actually can coexist within the primate visual system. Furthermore, we use principal component analysis to decompose stimulus information in population responses onto different principal dimensions in a high-dimensional representational space. Interestingly, response correlations simultaneously reduce and enhance information on higher- and lower-variance principal dimensions, respectively. The relative strength of the two antagonistic effects within the same computational framework produces the apparent discrepancy in the effects of response correlations in neuronal and voxel populations. Our results suggest that multivariate fMRI data contain rich statistical structures that are directly related to sensory information representation, and the general computational framework to analyze neuronal and voxel population responses can be applied in many types of neural measurements.SIGNIFICANCE STATEMENT Despite the vast research interest in the effect of spike-count noise correlations on population codes in neurophysiology, it remains unclear how the response correlations between voxels influence MVPA in human imaging. We used an information-theoretic approach and showed that unlike the detrimental effects of response correlations reported in neurophysiology, voxelwise response correlations generally improve sensory coding. We conducted a series of in-depth analyses and demonstrated that neuronal and voxel response correlations can coexist within the visual system and share some common computational mechanisms. These results shed new light on how the population codes of sensory information can be evaluated via different neural measurements.

Improving the validity of neuroimaging decoding tests of invariant and configural neural representation

Many research questions in sensory neuroscience involve determining whether the neural representation of a stimulus property is invariant or specific to a particular stimulus context (e.g., Is object representation invariant to translation? Is the representation of a face feature specific to the context of other face features?). Between these two extremes, representations may also be context-tolerant or context-sensitive. Most neuroimaging studies have used operational tests in which a target property is inferred from a significant test against the null hypothesis of the opposite property. For example, the popular cross-classification test concludes that representations are invariant or tolerant when the null hypothesis of specificity is rejected. A recently developed neurocomputational theory suggests two insights regarding such tests. First, tests against the null of context-specificity, and for the alternative of context-invariance, are prone to false positives due to the way in which the underlying neural representations are transformed into indirect measurements in neuroimaging studies. Second, jointly performing tests against the nulls of invariance and specificity allows one to reach more precise and valid conclusions about the underlying representations, particularly when the null of invariance is tested using the fine-grained information from classifier decision variables rather than only accuracies (i.e., using the decoding separability test). Here, we provide empirical and computational evidence supporting both of these theoretical insights. In our empirical study, we use encoding of orientation and spatial position in primary visual cortex as a case study, as previous research has established that these properties are encoded in a context-sensitive way. Using fMRI decoding, we show that the cross-classification test produces false-positive conclusions of invariance, but that more valid conclusions can be reached by jointly performing tests against the null of invariance. The results of two simulations further support both of these conclusions. We conclude that more valid inferences about invariance or specificity of neural representations can be reached by jointly testing against both hypotheses, and using neurocomputational theory to guide the interpretation of results.

A critical assessment of data quality and venous effects in sub-millimeter fMRI

Advances in hardware, pulse sequences, and reconstruction techniques have made it possible to perform functional magnetic resonance imaging (fMRI) at sub-millimeter resolution while maintaining high spatial coverage and acceptable signal-to-noise ratio. Here, we examine whether sub-millimeter fMRI can be used as a routine method for obtaining accurate measurements of fine-scale local neural activity. We conducted fMRI in human visual cortex during a simple event-related visual experiment (7 T, gradient-echo EPI, 0.8-mm isotropic voxels, 2.2-s sampling rate, 84 slices), and developed analysis and visualization tools to assess the quality of the data. Our results fall along three lines of inquiry. First, we find that the acquired fMRI images, combined with appropriate surface-based processing, provide reliable and accurate measurements of fine-scale blood oxygenation level dependent (BOLD) activity patterns. Second, we show that the highly folded structure of cortex causes substantial biases on spatial resolution and data visualization. Third, we examine the well-recognized issue of venous contributions to fMRI signals. In a systematic assessment of large sections of cortex measured at a fine scale, we show that time-averaged T2*-weighted EPI intensity is a simple, robust marker of venous effects. These venous effects are unevenly distributed across cortex, are more pronounced in gyri and outer cortical depths, and are, to a certain degree, in consistent locations across subjects relative to cortical folding. Furthermore, we show that these venous effects are strongly correlated with BOLD responses evoked by the experiment. We conclude that sub-millimeter fMRI can provide robust information about fine-scale BOLD activity patterns, but special care must be exercised in visualizing and interpreting these patterns, especially with regards to the confounding influence of the brain's vasculature. To help translate these methodological findings to neuroscience research, we provide practical suggestions for both high-resolution and standard-resolution fMRI studies.

Representation of Auditory Motion Directions and Sound Source Locations in the Human Planum Temporale

The ability to compute the location and direction of sounds is a crucial perceptual skill to efficiently interact with dynamic environments. How the human brain implements spatial hearing is, however, poorly understood. In our study, we used fMRI to characterize the brain activity of male and female humans listening to sounds moving left, right, up, and down as well as static sounds. Whole-brain univariate results contrasting moving and static sounds varying in their location revealed a robust functional preference for auditory motion in bilateral human planum temporale (hPT). Using independently localized hPT, we show that this region contains information about auditory motion directions and, to a lesser extent, sound source locations. Moreover, hPT showed an axis of motion organization reminiscent of the functional organization of the middle-temporal cortex (hMT+/V5) for vision. Importantly, whereas motion direction and location rely on partially shared pattern geometries in hPT, as demonstrated by successful cross-condition decoding, the responses elicited by static and moving sounds were, however, significantly distinct. Altogether, our results demonstrate that the hPT codes for auditory motion and location but that the underlying neural computation linked to motion processing is more reliable and partially distinct from the one supporting sound source location.SIGNIFICANCE STATEMENT Compared with what we know about visual motion, little is known about how the brain implements spatial hearing. Our study reveals that motion directions and sound source locations can be reliably decoded in the human planum temporale (hPT) and that they rely on partially shared pattern geometries. Our study, therefore, sheds important new light on how computing the location or direction of sounds is implemented in the human auditory cortex by showing that those two computations rely on partially shared neural codes. Furthermore, our results show that the neural representation of moving sounds in hPT follows a "preferred axis of motion" organization, reminiscent of the coding mechanisms typically observed in the occipital middle-temporal cortex (hMT+/V5) region for computing visual motion.

Stimulus vignetting and orientation selectivity in human visual cortex

Neural selectivity to orientation is one of the simplest and most thoroughly-studied cortical sensory features. Here, we show that a large body of research that purported to measure orientation tuning may have in fact been inadvertently measuring sensitivity to second-order changes in luminance, a phenomenon we term 'vignetting'. Using a computational model of neural responses in primary visual cortex (V1), we demonstrate the impact of vignetting on simulated V1 responses. We then used the model to generate a set of predictions, which we confirmed with functional MRI experiments in human observers. Our results demonstrate that stimulus vignetting can wholly determine the orientation selectivity of responses in visual cortex measured at a macroscopic scale, and suggest a reinterpretation of a well-established literature on orientation processing in visual cortex.

Just above Chance: Is It Harder to Decode Information from Prefrontal Cortex Hemodynamic Activity Patterns?

The prefrontal cortex (PFC) is central to flexible, goal-directed cognition, and understanding its representational code is an important problem in cognitive neuroscience. In humans, multivariate pattern analysis (MVPA) of fMRI blood oxygenation level-dependent (BOLD) measurements has emerged as an important approach for studying neural representations. Many previous studies have implicitly assumed that MVPA of fMRI BOLD is just as effective in decoding information encoded in PFC neural activity as it is in visual cortex. However, MVPA studies of PFC have had mixed success. Here we estimate the base rate of decoding information from PFC BOLD activity patterns from a meta-analysis of published MVPA studies. We show that PFC has a significantly lower base rate (55.4%) than visual areas in occipital (66.6%) and temporal (71.0%) cortices and one that is close to chance levels. Our results have implications for the design and interpretation of MVPA studies of PFC and raise important questions about its functional organization.

Spatial band-pass filtering aids decoding musical genres from auditory cortex 7T fMRI

Spatial filtering strategies, combined with multivariate decoding analysis of BOLD images, have been used to investigate the nature of the neural signal underlying the discriminability of brain activity patterns evoked by sensory stimulation – primarily in the visual cortex. Previous research indicates that such signals are spatially broadband in nature, and are not primarily comprised of fine-grained activation patterns. However, it is unclear whether this is a general property of the BOLD signal, or whether it is specific to the details of employed analyses and stimuli. Here we applied an analysis strategy from a previous study on decoding visual orientation from V1 to publicly available, high-resolution 7T fMRI on the response BOLD response to musical genres in primary auditory cortex. The results show that the pattern of decoding accuracies with respect to different types and levels of spatial filtering is comparable to that obtained from V1, despite considerable differences in the respective cortical circuitry.

Effects of spatial fMRI resolution on the classification of naturalistic movies

Studies involving multivariate pattern analysis (MVPA) of BOLD fMRI data generally attribute the success of the information-theoretic approach to BOLD signal contrast on the fine spatial scale of millimeters facilitating the classification or decoding of perceptual stimuli. However, to date MVPA studies that have actually explored fMRI resolutions at less than 2 mm voxel size are rare and limited to small sets of unnatural stimuli (like visual gratings) as well as specific sub-regions of the brain, notably the primary somatosensory cortices. To investigate what spatial scale best supports high information extraction under more general conditions this study combined naturalistic movie stimuli with high-resolution fMRI at 7 T and linear discriminant analysis (LDA) of global and local BOLD signal patterns. Contrary to predictions, LDA and similar classifiers reached a maximum in classification accuracy (CA) at a smoothed resolution close to 3 mm, well above the 1.2 mm voxel size of the fMRI acquisition. Maximal CAs around 90% were contingent upon global fMRI signal patterns comprising 4 k-16 k of the most reactive voxels distributed sparsely throughout the occipital and ventro-temporal cortices. A Searchlight analysis of local fMRI patterns largely confirmed the global results, but also revealed a small subset of brain regions in early visual cortex showing limited increases in CA with higher resolution. Principal component analysis of the global and local fMRI signal patterns suggested that reproducible neuronal contributions were spatially auto-correlated and smooth, while other components of higher spatial frequency were likely related to physiological noise and responsible for the reduced CA at higher resolution. Systematic differences between experiments and subjects suggested that higher CA was significantly correlated with more consistent behavior revealed by eye tracking. Thus, the optimal resolution of fMRI data for MVPA was mainly limited by physiological noise of high spatial frequency as well as behavioral (in-)consistency.

Ultra high-field (7 T) multi-resolution fMRI data for orientation decoding in visual cortex

Multivariate pattern classification methods have been successfully applied to decode orientation of visual grating stimuli from BOLD fMRI activity recorded in human visual cortex (Kamitani and Tong, 2005; Haynes and Rees, 2005) [12], [10]. Though there has been extensive research investigating the true spatial scale of the orientation specific signals (Op de Beeck, 2010; Swisher et al., 2010; Alink et al., 2013; Freeman et al., 2011, 2013) [2], [15], [1], [4], [5], it remained inconclusive what spatial acquisition resolution is required, or is optimal, for decoding analyses. The research article entitled "The effect of acquisition resolution on orientation decoding from V1 BOLD fMRI at 7 T" Sengupta et al. (2017) [14] studied the effect of spatial acquisition resolution and also analyzed the strength and spatial scale of orientation discriminating signals. In this article, for the first time, we present empirical ultra high-field fMRI data, obtained as a part of the aforementioned study, which were recorded at four spatial resolutions (0.8 mm, 1.4 mm, 2 mm, and 3 mm isotropic voxel size) for orientation decoding in visual cortex. The dataset is compliant with the BIDS (Brain Imaging Data Structure) format, and freely available from the OpenfMRI portal (dataset accession number: http://openfmri.org/dataset/ds000113c ds000113c).

The Effect of Spatial Smoothing on Representational Similarity in a Simple Motor Paradigm

Multi-voxel pattern analyses (MVPA) are often performed on unsmoothed data, which is very different from the general practice of large smoothing extents in standard voxel-based analyses. In this report, we studied the effect of smoothing on MVPA results in a motor paradigm. Subjects pressed four buttons with two different fingers of the two hands in response to auditory commands. Overall, independent of the degree of smoothing, correlational MVPA showed distinctive patterns for the different hands in all studied regions of interest (motor cortex, prefrontal cortex, and auditory cortices). With regard to the effect of smoothing, our findings suggest that results from correlational MVPA show a minor sensitivity to smoothing. Moderate amounts of smoothing (in this case, 1−4 times the voxel size) improved MVPA correlations, from a slight improvement to large improvements depending on the region involved. None of the regions showed signs of a detrimental effect of moderate levels of smoothing. Even higher amounts of smoothing sometimes had a positive effect, most clearly in low-level auditory cortex. We conclude that smoothing seems to have a minor positive effect on MVPA results, thus researchers should be mindful about the choices they make regarding the level of smoothing.

Optimization of functional MRI for detection, decoding and high-resolution imaging of the response patterns of cortical columns

The capacity of functional MRI (fMRI) to resolve cortical columns depends on several factors. These include the spatial scale of the columnar pattern, the point-spread of the fMRI response, the voxel size, and the signal-to-noise ratio (SNR) considering thermal and physiological noise. However, it remains unknown how these factors combine, and what is the voxel size that optimizes fMRI of cortical columns. Here we combine current knowledge into a quantitative model of fMRI of realistic patterns of cortical columns with different spatial scales and degrees of irregularity. We compare different approaches for identifying patterns of cortical columns, including univariate and multivariate based detection, multi-voxel pattern analysis (MVPA) based decoding, and high-resolution imaging and reconstruction of the pattern of cortical columns. We present the dependence of the performance of each approach on the parameters of the imaged pattern as well as those of the data acquisition. In addition, we predict voxel sizes that optimize fMRI of cortical columns under various scenarios. We found that all measures associated with multivariate detection and decoding could be approximately calculated from a measure we termed "multivariate contrast-to-noise ratio" (mv-CNR), which is a function of the contrast-to-noise ratio (CNR) and number of voxels. Furthermore, mv-CNR implied that the optimal voxel width for detection and decoding is independent of changes in response amplitude, SNR and imaged volume that are not caused by changes in voxel size. For regular patterns, optimal voxel widths for detection, decoding and imaging/reconstructing the pattern of cortical columns were approximately half the main cycle length of the organization. Optimal voxel widths for irregular patterns were less dependent on the main cycle length, and differed between univariate detection, multivariate detection and decoding, and reconstruction. We compared the effects of different factors of Gradient Echo fMRI at 3 Tesla (T), Gradient Echo fMRI at 7T, and Spin-Echo fMRI at 7T on the detection, decoding, and reconstruction measures considered and found that in all cases the width of the fMRI point-spread had the most significant effect. In contrast, different response amplitudes and noise characteristics played a relatively minor role. We recommend specific voxel widths for optimal univariate detection, for multivariate detection and decoding, and for high-resolution imaging of cortical columns under these three data-acquisition scenarios. Our study supports the planning, optimization, and interpretation of high-resolution fMRI of cortical columns and the decoding of information conveyed by these columns.