Improved decoding of neural activity from fMRI signals using non-separable spatiotemporal deconvolutions

Comparison of hemodynamic response functions obtained from resting-state functional MRI and invasive electrophysiological recordings in rats

Resting-state functional MRI (rs-fMRI) is a popular technology that has enriched our understanding of brain and spinal cord functioning, including how different regions communicate (connectivity). But fMRI is an indirect measure of neural activity capturing blood hemodynamics. The hemodynamic response function (HRF) interfaces between the unmeasured neural activity and measured fMRI time series. The HRF is variable across brain regions and individuals, and is modulated by non-neural factors. Ignoring this HRF variability causes errors in FC estimates. Hence, it is crucial to reliably estimate the HRF from rs-fMRI data. Robust techniques have emerged to estimate the HRF from fMRI time series. Although such techniques have been validated non-invasively using simulated and empirical fMRI data, thorough invasive validation using simultaneous electrophysiological recordings, the gold standard, has been elusive. This report addresses this gap in the literature by comparing HRFs derived from invasive intracranial electroencephalogram recordings with HRFs estimated from simultaneously acquired fMRI data in six epileptic rats. We found that the HRF shape parameters (HRF amplitude, latency and width) were not significantly different (p>0.05) between ground truth and estimated HRFs. In the single pathological region, the HRF width was marginally significantly different (p=0.03). Our study provides preliminary invasive validation for the efficacy of the HRF estimation technique in reliably estimating the HRF non-invasively from rs-fMRI data directly. This has a notable impact on rs-fMRI connectivity studies, and we recommend that HRF deconvolution be performed to minimize HRF variability and improve connectivity estimates.

Global effects of aging on the hemodynamic response function in the human brain

In functional magnetic resonance imaging, the hemodynamic response function (HRF) is a transient, stereotypical response to local changes in cerebral hemodynamics and oxygen metabolism due to briefly (< 4 s) evoked neural activity. Accordingly, the HRF is often used as an impulse response with the assumption of linearity in data analysis. In cognitive aging studies, it has been very common to interpret differences in brain activation as age-related changes in neural activity. Contrary to this assumption, however, evidence has accrued that normal aging may also significantly affect the vasculature, thereby affecting cerebral hemodynamics and metabolism, confounding interpretation of fMRI aging studies. In this study, use was made of a multisensory stimulus to evoke the HRF in ~ 87% of cerebral cortex in cognitively intact adults with ages ranging from 22-75 years. The stimulus evokes both positive and negative HRFs, which were characterized using model-free parameters in native-space coordinates. Results showed significant age trends in HRF parameter distributions in terms of both amplitudes (e.g., peak amplitude and CNR) and temporal dynamics (e.g., full-width-at-half-maximum). This work sets the stage for using HRF methods as a biomarker for age-related pathology.

The confound of hemodynamic response function variability in human resting-state functional MRI studies

Functional magnetic resonance imaging (fMRI) is an indirect measure of neural activity with the hemodynamic response function (HRF) coupling it with unmeasured neural activity. The HRF, modulated by several non-neural factors, is variable across brain regions, individuals and populations. Yet, a majority of human resting-state fMRI connectivity studies continue to assume a non-variable HRF. In this article, with supportive prior evidence, we argue that HRF variability cannot be ignored as it substantially confounds within-subject connectivity estimates and between-subjects connectivity group differences. We also discuss its clinical relevance with connectivity impairments confounded by HRF aberrations in several disorders. We present limited data on HRF differences between women and men, which resulted in a 15.4% median error in functional connectivity estimates in a group-level comparison. We also discuss the implications of HRF variability for fMRI studies in the spinal cord. There is a need for more dialogue within the community on the HRF confound, and we hope that our article is a catalyst in the process.

Canonical Response Parameterization: Quantifying the structure of responses to single-pulse intracranial electrical brain stimulation

Single-pulse electrical stimulation in the nervous system, often called cortico-cortical evoked potential (CCEP) measurement, is an important technique to understand how brain regions interact with one another. Voltages are measured from implanted electrodes in one brain area while stimulating another with brief current impulses separated by several seconds. Historically, researchers have tried to understand the significance of evoked voltage polyphasic deflections by visual inspection, but no general-purpose tool has emerged to understand their shapes or describe them mathematically. We describe and illustrate a new technique to parameterize brain stimulation data, where voltage response traces are projected into one another using a semi-normalized dot product. The length of timepoints from stimulation included in the dot product is varied to obtain a temporal profile of structural significance, and the peak of the profile uniquely identifies the duration of the response. Using linear kernel PCA, a canonical response shape is obtained over this duration, and then single-trial traces are parameterized as a projection of this canonical shape with a residual term. Such parameterization allows for dissimilar trace shapes from different brain areas to be directly compared by quantifying cross-projection magnitudes, response duration, canonical shape projection amplitudes, signal-to-noise ratios, explained variance, and statistical significance. Artifactual trials are automatically identified by outliers in sub-distributions of cross-projection magnitude, and rejected. This technique, which we call "Canonical Response Parameterization" (CRP) dramatically simplifies the study of CCEP shapes, and may also be applied in a wide range of other settings involving event-triggered data.

Interpretation for Individual Brain Age Prediction Based on Gray Matter Volume

The relationship between age and the central nervous system (CNS) in humans has been a classical issue that has aroused extensive attention. Especially for individuals, it is of far greater importance to clarify the mechanisms between CNS and age. The primary goal of existing methods is to use MR images to derive high-accuracy predictions for age or degenerative diseases. However, the associated mechanisms between the images and the age have rarely been investigated. In this paper, we address the correlation between gray matter volume (GMV) and age, both in terms of gray matter themselves and their interaction network, using interpretable machine learning models for individuals. Our goal is not only to predict age accurately but more importantly, to explore the relationship between GMV and age. In addition to targeting each individual, we also investigate the dynamic properties of gray matter and their interaction network with individual age. The results show that the mean absolute error (MAE) of age prediction is 7.95 years. More notably, specific locations of gray matter and their interactions play different roles in age, and these roles change dynamically with age. The proposed method is a data-driven approach, which provides a new way to study aging mechanisms and even to diagnose degenerative brain diseases.

FMRI hemodynamic response function (HRF) as a novel marker of brain function: applications for understanding obsessive-compulsive disorder pathology and treatment response

The hemodynamic response function (HRF) represents the transfer function linking neural activity with the functional MRI (fMRI) signal, modeling neurovascular coupling. Since HRF is influenced by non-neural factors, to date it has largely been considered as a confound or has been ignored in many analyses. However, underlying biophysics suggests that the HRF may contain meaningful correlates of neural activity, which might be unavailable through conventional fMRI metrics. Here, we estimated the HRF by performing deconvolution on resting-state fMRI data from a longitudinal sample of 25 healthy controls scanned twice and 44 adults with obsessive-compulsive disorder (OCD) before and after 4-weeks of intensive cognitive-behavioral therapy (CBT). HRF response height, time-to-peak and full-width at half-maximum (FWHM) in OCD were abnormal before treatment and normalized after treatment in regions including the caudate. Pre-treatment HRF predicted treatment outcome (OCD symptom reduction) with 86.4% accuracy, using machine learning. Pre-treatment HRF response height in the caudate head and time-to-peak in the caudate tail were top-predictors of treatment response. Time-to-peak in the caudate tail, a region not typically identified in OCD studies using conventional fMRI activation or connectivity measures, may carry novel importance. Additionally, pre-treatment response height in caudate head predicted post-treatment OCD severity (R = -0.48, P = 0.001), and was associated with treatment-related OCD severity changes (R = -0.44, P = 0.0028), underscoring its relevance. With HRF being a reliable marker sensitive to brain function, OCD pathology, and intervention-related changes, these results could guide future studies towards novel discoveries not possible through conventional fMRI approaches like standard BOLD activation or connectivity.

Decoding and interpreting cortical signals with a compact convolutional neural network

OBJECTIVE

Brain-computer interfaces (BCIs) decode information from neural activity and send it to external devices. The use of Deep Learning approaches for decoding allows for automatic feature engineering within the specific decoding task. Physiologically plausible interpretation of the network parameters ensures the robustness of the learned decision rules and opens the exciting opportunity for automatic knowledge discovery.

APPROACH

We describe a compact convolutional network-based architecture for adaptive decoding of electrocorticographic (ECoG) data into finger kinematics. We also propose a novel theoretically justified approach to interpreting the spatial and temporal weights in the architectures that combine adaptation in both space and time. The obtained spatial and frequency patterns characterizing the neuronal populations pivotal to the specific decoding task can then be interpreted by fitting appropriate spatial and dynamical models.

MAIN RESULTS

We first tested our solution using realistic Monte-Carlo simulations. Then, when applied to the ECoG data from Berlin BCI competition IV dataset, our architecture performed comparably to the competition winners without requiring explicit feature engineering. Using the proposed approach to the network weights interpretation we could unravel the spatial and the spectral patterns of the neuronal processes underlying the successful decoding of finger kinematics from an ECoG dataset. Finally we have also applied the entire pipeline to the analysis of a 32-channel EEG motor-imagery dataset and observed physiologically plausible patterns specific to the task.

SIGNIFICANCE

We described a compact and interpretable CNN architecture derived from the basic principles and encompassing the knowledge in the field of neural electrophysiology. For the first time in the context of such multibranch architectures with factorized spatial and temporal processing we presented theoretically justified weights interpretation rules. We verified our recipes using simulations and real data and demonstrated that the proposed solution offers a good decoder and a tool for investigating motor control neural mechanisms.

Holding On to the Past: Older Adults Show Lingering Neural Activation of No-Longer-Relevant Items in Working Memory

Goal-relevant information can be maintained in working memory over a brief delay interval to guide an upcoming decision. There is also evidence suggesting the existence of a complementary process: namely, the ability to suppress information that is no longer relevant to ongoing task goals. Moreover, this ability to suppress or inhibit irrelevant information appears to decline with age. In this study, we compared younger and older adults undergoing fMRI on a working memory task designed to address whether the modulation of neural representations of relevant and no-longer-relevant items during a delay interval is related to age and overall task performance. Following from the theoretical predictions of the inhibitory deficit hypothesis of aging, we hypothesized that older adults would show higher activation of no-longer-relevant items during a retention delay compared to young adults and that higher activation of these no-longer-relevant items would predict worse recognition memory accuracy for relevant items. Our results support this prediction and more generally demonstrate the importance of goal-driven modulation of neural activity in successful working memory maintenance. Furthermore, we showed that the largest age differences in the regulation of category-specific pattern activity during working memory maintenance were seen throughout the medial temporal lobe and prominently in the hippocampus, further establishing the importance of "long-term memory" retrieval mechanisms in the context of high-load working memory tasks that place large demands on attentional selection mechanisms.

Mechanical restriction of intracortical vessel dilation by brain tissue sculpts the hemodynamic response

Understanding the spatial dynamics of dilation in the cerebral vasculature is essential for deciphering the vascular basis of hemodynamic signals in the brain. We used two-photon microscopy to image neural activity and vascular dynamics in the somatosensory cortex of awake behaving mice during voluntary locomotion. Arterial dilations within the histologically-defined forelimb/hindlimb (FL/HL) representation were larger than arterial dilations in the somatosensory cortex immediately outside the FL/HL representation, demonstrating that the vascular response during natural behaviors was spatially localized. Surprisingly, we found that locomotion drove dilations in surface vessels that were nearly three times the amplitude of intracortical vessel dilations. The smaller dilations of the intracortical arterioles were not due to saturation of dilation. Anatomical imaging revealed that, unlike surface vessels, intracortical vessels were tightly enclosed by brain tissue. A mathematical model showed that mechanical restriction by the brain tissue surrounding intracortical vessels could account for the reduced amplitude of intracortical vessel dilation relative to surface vessels. Thus, under normal conditions, the mechanical properties of the brain may play an important role in sculpting the laminar differences of hemodynamic responses.

Quantitative separation of arterial and venous cerebral blood volume increases during voluntary locomotion

Voluntary locomotion is accompanied by large increases in cortical activity and localized increases in cerebral blood volume (CBV). We sought to quantitatively determine the spatial and temporal dynamics of voluntary locomotion-evoked cerebral hemodynamic changes. We measured single vessel dilations using two-photon microscopy and cortex-wide changes in CBV-related signal using intrinsic optical signal (IOS) imaging in head-fixed mice freely locomoting on a spherical treadmill. During bouts of locomotion, arteries dilated rapidly, while veins distended slightly and recovered slowly. The dynamics of diameter changes of both vessel types could be captured using a simple linear convolution model. Using these single vessel measurements, we developed a novel analysis approach to separate out spatially and temporally distinct arterial and venous components of the location-specific hemodynamic response functions (HRF) for IOS. The HRF of each pixel of was well fit by a sum of a fast arterial and a slow venous component. The HRFs of pixels in the limb representations of somatosensory cortex had a large arterial contribution, while in the frontal cortex the arterial contribution to the HRF was negligible. The venous contribution was much less localized, and was substantial in the frontal cortex. The spatial pattern and amplitude of these HRFs in response to locomotion in the cortex were robust across imaging sessions. Separating the more localized arterial component from the diffuse venous signals will be useful for dealing with the dynamic signals generated by naturalistic stimuli.

Deconvolution of neural dynamics from fMRI data using a spatiotemporal hemodynamic response function

Functional magnetic resonance imaging (fMRI) is a powerful and broadly used means of non-invasively mapping human brain activity. However fMRI is an indirect measure that rests upon a mapping from neuronal activity to the blood oxygen level dependent (BOLD) signal via hemodynamic effects. The quality of estimated neuronal activity hinges on the validity of the hemodynamic model employed. Recent work has demonstrated that the hemodynamic response has non-separable spatiotemporal dynamics, a key property that is not implemented in existing fMRI analysis frameworks. Here both simulated and empirical data are used to demonstrate that using a physiologically based model of the spatiotemporal hemodynamic response function (stHRF) results in a quantitative improvement of the estimated neuronal response relative to unphysical space-time separable forms. To achieve this, an integrated spatial and temporal deconvolution is established using a recently developed stHRF. Simulated data allows the variation of key parameters such as noise and the spatial complexity of the neuronal drive, while knowing the neuronal input. The results demonstrate that the use of a spatiotemporally integrated HRF can avoid "ghost" neuronal responses that can otherwise be falsely inferred. Applying the spatiotemporal deconvolution to high resolution fMRI data allows the recovery of neuronal responses that are consistent with independent electrophysiological measures.

On the interpretation of weight vectors of linear models in multivariate neuroimaging

The increase in spatiotemporal resolution of neuroimaging devices is accompanied by a trend towards more powerful multivariate analysis methods. Often it is desired to interpret the outcome of these methods with respect to the cognitive processes under study. Here we discuss which methods allow for such interpretations, and provide guidelines for choosing an appropriate analysis for a given experimental goal: For a surgeon who needs to decide where to remove brain tissue it is most important to determine the origin of cognitive functions and associated neural processes. In contrast, when communicating with paralyzed or comatose patients via brain-computer interfaces, it is most important to accurately extract the neural processes specific to a certain mental state. These equally important but complementary objectives require different analysis methods. Determining the origin of neural processes in time or space from the parameters of a data-driven model requires what we call a forward model of the data; such a model explains how the measured data was generated from the neural sources. Examples are general linear models (GLMs). Methods for the extraction of neural information from data can be considered as backward models, as they attempt to reverse the data generating process. Examples are multivariate classifiers. Here we demonstrate that the parameters of forward models are neurophysiologically interpretable in the sense that significant nonzero weights are only observed at channels the activity of which is related to the brain process under study. In contrast, the interpretation of backward model parameters can lead to wrong conclusions regarding the spatial or temporal origin of the neural signals of interest, since significant nonzero weights may also be observed at channels the activity of which is statistically independent of the brain process under study. As a remedy for the linear case, we propose a procedure for transforming backward models into forward models. This procedure enables the neurophysiological interpretation of the parameters of linear backward models. We hope that this work raises awareness for an often encountered problem and provides a theoretical basis for conducting better interpretable multivariate neuroimaging analyses.