A high-resolution 7-Tesla fMRI dataset from complex natural stimulation with an audio movie

A studyforrest extension, an annotation of spoken language in the German dubbed movie "Forrest Gump" and its audio-description

Here we present an annotation of speech in the audio-visual movie "Forrest Gump" and its audio-description for a visually impaired audience, as an addition to a large public functional brain imaging dataset ( studyforrest.org). The annotation provides information about the exact timing of each of the more than 2500 spoken sentences, 16,000 words (including 202 non-speech vocalizations), 66,000 phonemes, and their corresponding speaker. Additionally, for every word, we provide lemmatization, a simple part-of-speech-tagging (15 grammatical categories), a detailed part-of-speech tagging (43 grammatical categories), syntactic dependencies, and a semantic analysis based on word embedding which represents each word in a 300-dimensional semantic space. To validate the dataset's quality, we build a model of hemodynamic brain activity based on information drawn from the annotation. Results suggest that the annotation's content and quality enable independent researchers to create models of brain activity correlating with a variety of linguistic aspects under conditions of near-real-life complexity.

Subject-specific segregation of functional territories based on deep phenotyping

Functional magnetic resonance imaging (fMRI) has opened the possibility to investigate how brain activity is modulated by behavior. Most studies so far are bound to one single task, in which functional responses to a handful of contrasts are analyzed and reported as a group average brain map. Contrariwise, recent data-collection efforts have started to target a systematic spatial representation of multiple mental functions. In this paper, we leverage the Individual Brain Charting (IBC) dataset-a high-resolution task-fMRI dataset acquired in a fixed environment-in order to study the feasibility of individual mapping. First, we verify that the IBC brain maps reproduce those obtained from previous, large-scale datasets using the same tasks. Second, we confirm that the elementary spatial components, inferred across all tasks, are consistently mapped within and, to a lesser extent, across participants. Third, we demonstrate the relevance of the topographic information of the individual contrast maps, showing that contrasts from one task can be predicted by contrasts from other tasks. At last, we showcase the benefit of contrast accumulation for the fine functional characterization of brain regions within a prespecified network. To this end, we analyze the cognitive profile of functional territories pertaining to the language network and prove that these profiles generalize across participants.

Searching through functional space reveals distributed visual, auditory, and semantic coding in the human brain

The extent to which brain functions are localized or distributed is a foundational question in neuroscience. In the human brain, common fMRI methods such as cluster correction, atlas parcellation, and anatomical searchlight are biased by design toward finding localized representations. Here we introduce the functional searchlight approach as an alternative to anatomical searchlight analysis, the most commonly used exploratory multivariate fMRI technique. Functional searchlight removes any anatomical bias by grouping voxels based only on functional similarity and ignoring anatomical proximity. We report evidence that visual and auditory features from deep neural networks and semantic features from a natural language processing model, as well as object representations, are more widely distributed across the brain than previously acknowledged and that functional searchlight can improve model-based similarity and decoding accuracy. This approach provides a new way to evaluate and constrain computational models with brain activity and pushes our understanding of human brain function further along the spectrum from strict modularity toward distributed representation.

Keep it real: rethinking the primacy of experimental control in cognitive neuroscience

Naturalistic experimental paradigms in neuroimaging arose from a pressure to test the validity of models we derive from highly-controlled experiments in real-world contexts. In many cases, however, such efforts led to the realization that models developed under particular experimental manipulations failed to capture much variance outside the context of that manipulation. The critique of non-naturalistic experiments is not a recent development; it echoes a persistent and subversive thread in the history of modern psychology. The brain has evolved to guide behavior in a multidimensional world with many interacting variables. The assumption that artificially decoupling and manipulating these variables will lead to a satisfactory understanding of the brain may be untenable. We develop an argument for the primacy of naturalistic paradigms, and point to recent developments in machine learning as an example of the transformative power of relinquishing control. Naturalistic paradigms should not be deployed as an afterthought if we hope to build models of brain and behavior that extend beyond the laboratory into the real world.

An fMRI dataset in response to "The Grand Budapest Hotel", a socially-rich, naturalistic movie

Naturalistic stimuli evoke strong, consistent, and information-rich patterns of brain activity, and engage large extents of the human brain. They allow researchers to compare highly similar brain responses across subjects, and to study how complex representations are encoded in brain activity. Here, we describe and share a dataset where 25 subjects watched part of the feature film "The Grand Budapest Hotel" by Wes Anderson. The movie has a large cast with many famous actors. Throughout the story, the camera shots highlight faces and expressions, which are fundamental to understand the complex narrative of the movie. This movie was chosen to sample brain activity specifically related to social interactions and face processing. This dataset provides researchers with fMRI data that can be used to explore social cognitive processes and face processing, adding to the existing neuroimaging datasets that sample brain activity with naturalistic movies.

Individual Brain Charting dataset extension, second release of high-resolution fMRI data for cognitive mapping

We present an extension of the Individual Brain Charting dataset -a high spatial-resolution, multi-task, functional Magnetic Resonance Imaging dataset, intended to support the investigation on the functional principles governing cognition in the human brain. The concomitant data acquisition from the same 12 participants, in the same environment, allows to obtain in the long run finer cognitive topographies, free from inter-subject and inter-site variability. This second release provides more data from psychological domains present in the first release, and also yields data featuring new ones. It includes tasks on e.g. mental time travel, reward, theory-of-mind, pain, numerosity, self-reference effect and speech recognition. In total, 13 tasks with 86 contrasts were added to the dataset and 63 new components were included in the cognitive description of the ensuing contrasts. As the dataset becomes larger, the collection of the corresponding topographies becomes more comprehensive, leading to better brain-atlasing frameworks. This dataset is an open-access facility; raw data and derivatives are publicly available in neuroimaging repositories.

i-SATA: A MATLAB based toolbox to estimate current density generated by transcranial direct current stimulation in an individual brain

OBJECTIVE

Transcranial Direct Current Stimulation (tDCS) is a technique where a weak current is passed through the electrodes placed on the scalp. The distribution of the electric current induced in the brain due to tDCS is provided by simulation toolbox like Realistic volumetric Approach based Simulator for Transcranial electric stimulation (ROAST). However, the procedure to estimate the total current density induced at the target and the intermediary region of the cortex is complex. The Systematic-Approach-for-tDCS-Analysis (SATA) was developed to overcome this problem. However, SATA is limited to standardized (MNI152) headspace only. Here we develop individual-SATA (i-SATA) to extend it to individual head.

APPROACH

T1-weighted images of 15 subjects were taken from two Magnetic Resonance Imaging scanners of different strengths. Across the subjects, the montages were simulated in ROAST. i-SATA converts the ROAST output to Talairach space. The x, y and z coordinates of the anterior commissure (AC), posterior commissure (PC), and Mid-Sagittal (MS) points are necessary for the conversion. AC and PC are detected using the acpcdetect toolbox. We developed a method to determine the MS in the image and cross-verified its location manually using BrainSight®.

MAIN RESULTS

Determination of points with i-SATA is fast and accurate. The i-SATA provided estimates of the current-density induced across an individual's cortical lobes and gyri as tested on images from two different scanners.

SIGNIFICANCE

Researchers can use i-SATA for customizing tDCS-montages. With i-SATA it is also easier to compute the inter-individual variation in current-density across the target and intermediary regions of the brain. The software is publicly available.

A naturalistic neuroimaging database for understanding the brain using ecological stimuli

Neuroimaging has advanced our understanding of human psychology using reductionist stimuli that often do not resemble information the brain naturally encounters. It has improved our understanding of the network organization of the brain mostly through analyses of 'resting-state' data for which the functions of networks cannot be verifiably labelled. We make a 'Naturalistic Neuroimaging Database' (NNDb v1.0) publically available to allow for a more complete understanding of the brain under more ecological conditions during which networks can be labelled. Eighty-six participants underwent behavioural testing and watched one of 10 full-length movies while functional magnetic resonance imaging was acquired. Resulting timeseries data are shown to be of high quality, with good signal-to-noise ratio, few outliers and low movement. Data-driven functional analyses provide further evidence of data quality. They also demonstrate accurate timeseries/movie alignment and how movie annotations might be used to label networks. The NNDb can be used to answer questions previously unaddressed with standard neuroimaging approaches, progressing our knowledge of how the brain works in the real world.

Movies and narratives as naturalistic stimuli in neuroimaging

Using movies and narratives as naturalistic stimuli in human neuroimaging studies has yielded significant advances in understanding of cognitive and emotional functions. The relevant literature was reviewed, with emphasis on how the use of naturalistic stimuli has helped advance scientific understanding of human memory, attention, language, emotions, and social cognition in ways that would have been difficult otherwise. These advances include discovering a cortical hierarchy of temporal receptive windows, which supports processing of dynamic information that accumulates over several time scales, such as immediate reactions vs. slowly emerging patterns in social interactions. Naturalistic stimuli have also helped elucidate how the hippocampus supports segmentation and memorization of events in day-to-day life and have afforded insights into attentional brain mechanisms underlying our ability to adopt specific perspectives during natural viewing. Further, neuroimaging studies with naturalistic stimuli have revealed the role of the default-mode network in narrative-processing and in social cognition. Finally, by robustly eliciting genuine emotions, these stimuli have helped elucidate the brain basis of both basic and social emotions apparently manifested as highly overlapping yet distinguishable patterns of brain activity.

Leveraging shared connectivity to aggregate heterogeneous datasets into a common response space

Connectivity hyperalignment can be used to estimate a single shared response space across disjoint datasets. We develop a connectivity-based shared response model that factorizes aggregated fMRI datasets into a single reduced-dimension shared connectivity space and subject-specific topographic transformations. These transformations resolve idiosyncratic functional topographies and can be used to project response time series into shared space. We evaluate this algorithm on a large collection of heterogeneous, naturalistic fMRI datasets acquired while subjects listened to spoken stories. Projecting subject data into shared space dramatically improves between-subject story time-segment classification and increases the dimensionality of shared information across subjects. This improvement generalizes to subjects and stories excluded when estimating the shared space. We demonstrate that estimating a simple semantic encoding model in shared space improves between-subject forward encoding and inverted encoding model performance. The shared space estimated across all datasets is distinct from the shared space derived from any particular constituent dataset; the algorithm leverages shared connectivity to yield a consensus shared space conjoining diverse story stimuli.

Nature abhors a paywall: How open science can realize the potential of naturalistic stimuli

Naturalistic stimuli show significant potential to inform behavioral, cognitive, and clinical neuroscience. To date, this impact is still limited by the relative inaccessibility of both generated neuroimaging data as well as the supporting naturalistic stimuli. In this perspective, we highlight currently available naturalistic datasets and technical solutions such as DataLad that continue to advance our ability to share this data. We also review scientific and sociological challenges in selecting naturalistic stimuli for reproducible research. Overall, we encourage researchers to share their naturalistic datasets to the full extent possible under local copyright law.

Investigating time-varying functional connectivity derived from the Jackknife Correlation method for distinguishing between emotions in fMRI data

Investigating human brain activity during expressing emotional states provides deep insight into complex cognitive functions and neurological correlations inside the brain. To be able to resemble the brain function in the best manner, a complex and natural stimulus should be applied as well, the method used for data analysis should have fewer assumptions, simplifications, and parameter adjustment. In this study, we examined a functional magnetic resonance imaging dataset obtained during an emotional audio-movie stimulus associated with human life. We used Jackknife Correlation (JC) method to derive a representation of time-varying functional connectivity. We applied different binary measures and thoroughly investigated two weighted measures to study different properties of binary and weighted temporal networks. Using this approach, we indicated different aspects of human brain function during expressing different emotions. The findings of global and nodal measures could demonstrate a significant difference between emotions and significant regions in each emotion, respectively. Also, the temporal centrality properties of nodes were different in emotional states. Ultimately, we showed that the resulting measures of temporal snapshots created by JC method can distinguish between different emotions.

Predicting individual face-selective topography using naturalistic stimuli

Subject-specific, functionally defined areas are conventionally estimated with functional localizers and a simple contrast analysis between responses to different stimulus categories. Compared with functional localizers, naturalistic stimuli provide several advantages such as stronger and widespread brain activation, greater engagement, and increased subject compliance. In this study we demonstrate that a subject's idiosyncratic functional topography can be estimated with high fidelity from that subject's fMRI data obtained while watching a naturalistic movie using hyperalignment to project other subjects' localizer data into that subject's idiosyncratic cortical anatomy. These findings lay the foundation for developing an efficient tool for mapping functional topographies for a wide range of perceptual and cognitive functions in new subjects based only on fMRI data collected while watching an engaging, naturalistic stimulus and other subjects' localizer data from a normative sample.

A manually denoised audio-visual movie watching fMRI dataset for the studyforrest project

The data presented here are related to the studyforrest project that uses the movie ‘Forrest Gump’ to map brain functions in a real-life context using functional magnetic resonance imaging (fMRI). However, neural-related fMRI signals are often small and confounded by various noise sources (i.e., artifacts) that makes searching for the signals induced by specific cognitive processes significantly challenging. To make neural-related signals stand out from the noise, the audio-visual movie watching fMRI dataset from the project was denoised by a combination of spatial independent component analysis and manual identification of signals or noise. Here, both the denoised data and the labeled decomposed components are shared to facilitate further study. Compared with the original data, the denoised data showed a substantial improvement in the temporal signal-to-noise ratio and provided a higher sensitivity in subsequent analyses such as in an inter-subject correlation analysis.

Measurement(s)	Blood Oxygen Level-Dependent Functional MRI
Technology Type(s)	data transformation
Sample Characteristic - Organism	Homo sapiens

Machine-accessible metadata file describing the reported data: 10.6084/m9.figshare.10266554

Brain dynamics and connectivity networks under natural auditory stimulation

The analysis of functional magnetic resonance imaging (fMRI) data is challenging when subjects are under exposure to natural sensory stimulation. In this study, a two-stage approach was developed to enable the identification of connectivity networks involved in the processing of information in the brain under natural sensory stimulation. In the first stage, the degree of concordance between the results of inter-subject and intra-subject correlation analyses is assessed statistically. The microstructurally (i.e., cytoarchitectonically) defined brain areas are designated either as concordant in which the results of both correlation analyses are in agreement, or as discordant in which one analysis method shows a higher proportion of supra-threshold voxels than does the other. In the second stage, connectivity networks are identified using the time courses of supra-threshold voxels in brain areas contingent upon the classifications derived in the first stage. In an empirical study, fMRI data were collected from 40 young adults (19 males, average age 22.76 ± 3.25), who underwent auditory stimulation involving sound clips of human voices and animal vocalizations under two operational conditions (i.e., eyes-closed and eyes-open). The operational conditions were designed to assess confounding effects due to auditory instructions or visual perception. The proposed two-stage analysis demonstrated that stress modulation (affective) and language networks in the limbic and cortical structures were respectively engaged during sound stimulation, and presented considerable variability among subjects. The network involved in regulating visuomotor control was sensitive to the eyes-open instruction, and presented only small variations among subjects. A high degree of concordance was observed between the two analyses in the primary auditory cortex which was highly sensitive to the pitch of sound clips. Our results have indicated that brain areas can be identified as concordant or discordant based on the two correlation analyses. This may further facilitate the search for connectivity networks involved in the processing of information under natural sensory stimulation.

Multi-Objective Cognitive Model: a Supervised Approach for Multi-subject fMRI Analysis

In order to decode human brain, Multivariate Pattern (MVP) classification generates cognitive models by using functional Magnetic Resonance Imaging (fMRI) datasets. As a standard pipeline in the MVP analysis, brain patterns in multi-subject fMRI dataset must be mapped to a shared space and then a classification model is generated by employing the mapped patterns. However, the MVP models may not provide stable performance on a new fMRI dataset because the standard pipeline uses disjoint steps for generating these models. Indeed, each step in the pipeline includes an objective function with independent optimization approach, where the best solution of each step may not be optimum for the next steps. For tackling the mentioned issue, this paper introduces Multi-Objective Cognitive Model (MOCM) that utilizes an integrated objective function for MVP analysis rather than just using those disjoint steps. For solving the integrated problem, we proposed a customized multi-objective optimization approach, where all possible solutions are firstly generated, and then our method ranks and selects the robust solutions as the final results. Empirical studies confirm that the proposed method can generate superior performance in comparison with other techniques.

Naturalistic Stimuli in Neuroscience: Critically Acclaimed

Cognitive neuroscience has traditionally focused on simple tasks, presented sparsely and using abstract stimuli. While this approach has yielded fundamental insights into functional specialisation in the brain, its ecological validity remains uncertain. Do these tasks capture how brains function 'in the wild', where stimuli are dynamic, multimodal, and crowded? Ecologically valid paradigms that approximate real life scenarios, using stimuli such as films, spoken narratives, music, and multiperson games emerged in response to these concerns over a decade ago. We critically appraise whether this approach has delivered on its promise to deliver new insights into brain function. We highlight the challenges, technological innovations, and clinical opportunities that are required should this field meet its full potential.

Teaching Computational Reproducibility for Neuroimaging

We describe a project-based introduction to reproducible and collaborative neuroimaging analysis. Traditional teaching on neuroimaging usually consists of a series of lectures that emphasize the big picture rather than the foundations on which the techniques are based. The lectures are often paired with practical workshops in which students run imaging analyses using the graphical interface of specific neuroimaging software packages. Our experience suggests that this combination leaves the student with a superficial understanding of the underlying ideas, and an informal, inefficient, and inaccurate approach to analysis. To address these problems, we based our course around a substantial open-ended group project. This allowed us to teach: (a) computational tools to ensure computationally reproducible work, such as the Unix command line, structured code, version control, automated testing, and code review and (b) a clear understanding of the statistical techniques used for a basic analysis of a single run in an MR scanner. The emphasis we put on the group project showed the importance of standard computational tools for accuracy, efficiency, and collaboration. The projects were broadly successful in engaging students in working reproducibly on real scientific questions. We propose that a course on this model should be the foundation for future programs in neuroimaging. We believe it will also serve as a model for teaching efficient and reproducible research in other fields of computational science.

The Hippocampal Film Editor: Sensitivity and Specificity to Event Boundaries in Continuous Experience

The function of the human hippocampus is normally investigated by experimental manipulation of discrete events. Less is known about what triggers hippocampal activity during more naturalistic, continuous experience. We hypothesized that the hippocampus would be sensitive to the occurrence of event boundaries, that is, moments in time identified by observers as a transition between events. To address this, we analyzed functional MRI data from two groups: one (n = 253, 131 female) who viewed an 8.5 min film and another (n = 15, 6 female) who viewed a 120 min film. We observed a strong hippocampal response at boundaries defined by independent observers, which was modulated by boundary salience (the number of observers that identified each boundary). In the longer film, there were sufficient boundaries to show that this modulation remained after covarying out a large number of perceptual factors. This hypothesis-driven approach was complemented by a data-driven approach, in which we identified hippocampal events as moments in time with the strongest hippocampal activity. The correspondence between these hippocampal events and event boundaries was highly significant, revealing that the hippocampal response is not only sensitive, but also specific to event boundaries. We conclude that event boundaries play a key role in shaping hippocampal activity during encoding of naturalistic events.SIGNIFICANCE STATEMENT Recent years have seen the field of human neuroscience research transitioning from experiments with simple stimuli to the study of more complex and naturalistic experience. Nonetheless, our understanding of the function of many brain regions, such as the hippocampus, is based primarily on the study of brief, discrete events. As a result, we know little of what triggers hippocampal activity in real-life settings when we are exposed to a continuous stream of information. When does the hippocampus "decide" to respond during the encoding of naturalistic experience? We reveal here that hippocampal activity measured by fMRI during film watching is both sensitive and specific to event boundaries, identifying a potential mechanism whereby event boundaries shape experience by modulation of hippocampal activity.

Cooperating yet distinct brain networks engaged during naturalistic paradigms: A meta-analysis of functional MRI results

Cognitive processes do not occur by pure insertion and instead depend on the full complement of co-occurring mental processes, including perceptual and motor functions. As such, there is limited ecological validity to human neuroimaging experiments that use highly controlled tasks to isolate mental processes of interest. However, a growing literature shows how dynamic, interactive tasks have allowed researchers to study cognition as it more naturally occurs. Collective analysis across such neuroimaging experiments may answer broader questions regarding how naturalistic cognition is biologically distributed throughout the brain. We applied an unbiased, data-driven, meta-analytic approach that uses k-means clustering to identify core brain networks engaged across the naturalistic functional neuroimaging literature. Functional decoding allowed us to, then, delineate how information is distributed between these networks throughout the execution of dynamical cognition in realistic settings. This analysis revealed six recurrent patterns of brain activation, representing sensory, domain-specific, and attentional neural networks that support the cognitive demands of naturalistic paradigms. Although gaps in the literature remain, these results suggest that naturalistic fMRI paradigms recruit a common set of networks that allow both separate processing of different streams of information and integration of relevant information to enable flexible cognition and complex behavior.

Individual Brain Charting, a high-resolution fMRI dataset for cognitive mapping

Functional Magnetic Resonance Imaging (fMRI) has furthered brain mapping on perceptual, motor, as well as higher-level cognitive functions. However, to date, no data collection has systematically addressed the functional mapping of cognitive mechanisms at a fine spatial scale. The Individual Brain Charting (IBC) project stands for a high-resolution multi-task fMRI dataset that intends to provide the objective basis toward a comprehensive functional atlas of the human brain. The data refer to a cohort of 12 participants performing many different tasks. The large amount of task-fMRI data on the same subjects yields a precise mapping of the underlying functions, free from both inter-subject and inter-site variability. The present article gives a detailed description of the first release of the IBC dataset. It comprises a dozen of tasks, addressing both low- and high- level cognitive functions. This openly available dataset is thus intended to become a reference for cognitive brain mapping.

Hypersampling of pseudo-periodic signals by analytic phase projection

A method to upsample insufficiently sampled experimental time series of pseudo-periodic signals is proposed. The result is an estimate of the pseudo-periodic cycle underlying the signal. This "hypersampling" requires a sufficiently sampled reference signal that defines the pseudo-periodic dynamics. The time series and reference signal are combined by projecting the time series values to the analytic phase of the reference signal. The resulting estimate of the pseudo-periodic cycle has a considerably higher effective sampling rate than the time series. The procedure is applied to time series of MRI images of the human brain. As a result, the effective sampling rate could be increased by three orders of magnitude. This allows for capturing the waveforms of the very fast cerebral pulse waves traversing the brain. Hypersampling is numerically compared to the more commonly used retrospective gating. An outlook regarding EEG and optical recordings of brain activity as the reference signal is provided.

Spatial band-pass filtering aids decoding musical genres from auditory cortex 7T fMRI

Spatial filtering strategies, combined with multivariate decoding analysis of BOLD images, have been used to investigate the nature of the neural signal underlying the discriminability of brain activity patterns evoked by sensory stimulation – primarily in the visual cortex. Previous research indicates that such signals are spatially broadband in nature, and are not primarily comprised of fine-grained activation patterns. However, it is unclear whether this is a general property of the BOLD signal, or whether it is specific to the details of employed analyses and stimuli. Here we applied an analysis strategy from a previous study on decoding visual orientation from V1 to publicly available, high-resolution 7T fMRI on the response BOLD response to musical genres in primary auditory cortex. The results show that the pattern of decoding accuracies with respect to different types and levels of spatial filtering is comparable to that obtained from V1, despite considerable differences in the respective cortical circuitry.

Decoding power-spectral profiles from FMRI brain activities during naturalistic auditory experience

Recent studies have demonstrated a close relationship between computational acoustic features and neural brain activities, and have largely advanced our understanding of auditory information processing in the human brain. Along this line, we proposed a multidisciplinary study to examine whether power spectral density (PSD) profiles can be decoded from brain activities during naturalistic auditory experience. The study was performed on a high resolution functional magnetic resonance imaging (fMRI) dataset acquired when participants freely listened to the audio-description of the movie "Forrest Gump". Representative PSD profiles existing in the audio-movie were identified by clustering the audio samples according to their PSD descriptors. Support vector machine (SVM) classifiers were trained to differentiate the representative PSD profiles using corresponding fMRI brain activities. Based on PSD profile decoding, we explored how the neural decodability correlated to power intensity and frequency deviants. Our experimental results demonstrated that PSD profiles can be reliably decoded from brain activities. We also suggested a sigmoidal relationship between the neural decodability and power intensity deviants of PSD profiles. Our study in addition substantiates the feasibility and advantage of naturalistic paradigm for studying neural encoding of complex auditory information.

Electrophysiology Reveals the Neural Dynamics of Naturalistic Auditory Language Processing: Event-Related Potentials Reflect Continuous Model Updates

The recent trend away from ANOVA-based analyses places experimental investigations into the neurobiology of cognition in more naturalistic and ecologically valid designs within reach. Using mixed-effects models for epoch-based regression, we demonstrate the feasibility of examining event-related potentials (ERPs), and in particular the N400, to study the neural dynamics of human auditory language processing in a naturalistic setting. Despite the large variability between trials during naturalistic stimulation, we replicated previous findings from the literature: the effects of frequency, animacy, and word order and find previously unexplored interaction effects. This suggests a new perspective on ERPs, namely, as a continuous modulation reflecting continuous stimulation instead of a series of discrete and essentially sequential processes locked to discrete events.

Ultra high-field (7 T) multi-resolution fMRI data for orientation decoding in visual cortex

Multivariate pattern classification methods have been successfully applied to decode orientation of visual grating stimuli from BOLD fMRI activity recorded in human visual cortex (Kamitani and Tong, 2005; Haynes and Rees, 2005) [12], [10]. Though there has been extensive research investigating the true spatial scale of the orientation specific signals (Op de Beeck, 2010; Swisher et al., 2010; Alink et al., 2013; Freeman et al., 2011, 2013) [2], [15], [1], [4], [5], it remained inconclusive what spatial acquisition resolution is required, or is optimal, for decoding analyses. The research article entitled "The effect of acquisition resolution on orientation decoding from V1 BOLD fMRI at 7 T" Sengupta et al. (2017) [14] studied the effect of spatial acquisition resolution and also analyzed the strength and spatial scale of orientation discriminating signals. In this article, for the first time, we present empirical ultra high-field fMRI data, obtained as a part of the aforementioned study, which were recorded at four spatial resolutions (0.8 mm, 1.4 mm, 2 mm, and 3 mm isotropic voxel size) for orientation decoding in visual cortex. The dataset is compliant with the BIDS (Brain Imaging Data Structure) format, and freely available from the OpenfMRI portal (dataset accession number: http://openfmri.org/dataset/ds000113c ds000113c).

Music of the 7Ts: Predicting and Decoding Multivoxel fMRI Responses with Acoustic, Schematic, and Categorical Music Features

Underlying the experience of listening to music are parallel streams of auditory, categorical, and schematic qualia, whose representations and cortical organization remain largely unresolved. We collected high-field (7T) fMRI data in a music listening task, and analyzed the data using multivariate decoding and stimulus-encoding models. Twenty subjects participated in the experiment, which measured BOLD responses evoked by naturalistic listening to twenty-five music clips from five genres. Our first analysis applied machine classification to the multivoxel patterns that were evoked in temporal cortex. Results yielded above-chance levels for both stimulus identification and genre classification–cross-validated by holding out data from multiple of the stimuli during model training and then testing decoding performance on the held-out data. Genre model misclassifications were significantly correlated with those in a corresponding behavioral music categorization task, supporting the hypothesis that geometric properties of multivoxel pattern spaces underlie observed musical behavior. A second analysis employed a spherical searchlight regression analysis which predicted multivoxel pattern responses to music features representing melody and harmony across a large area of cortex. The resulting prediction-accuracy maps yielded significant clusters in the temporal, frontal, parietal, and occipital lobes, as well as in the parahippocampal gyrus and the cerebellum. These maps provide evidence in support of our hypothesis that geometric properties of music cognition are neurally encoded as multivoxel representational spaces. The maps also reveal a cortical topography that differentially encodes categorical and absolute-pitch information in distributed and overlapping networks, with smaller specialized regions that encode tonal music information in relative-pitch representations.

Functional brain segmentation using inter-subject correlation in fMRI

The human brain continuously processes massive amounts of rich sensory information. To better understand such highly complex brain processes, modern neuroimaging studies are increasingly utilizing experimental setups that better mimic daily-life situations. A new exploratory data-analysis approach, functional segmentation inter-subject correlation analysis (FuSeISC), was proposed to facilitate the analysis of functional magnetic resonance (fMRI) data sets collected in these experiments. The method provides a new type of functional segmentation of brain areas, not only characterizing areas that display similar processing across subjects but also areas in which processing across subjects is highly variable. FuSeISC was tested using fMRI data sets collected during traditional block-design stimuli (37 subjects) as well as naturalistic auditory narratives (19 subjects). The method identified spatially local and/or bilaterally symmetric clusters in several cortical areas, many of which are known to be processing the types of stimuli used in the experiments. The method is not only useful for spatial exploration of large fMRI data sets obtained using naturalistic stimuli, but also has other potential applications, such as generation of a functional brain atlases including both lower- and higher-order processing areas. Finally, as a part of FuSeISC, a criterion-based sparsification of the shared nearest-neighbor graph was proposed for detecting clusters in noisy data. In the tests with synthetic data, this technique was superior to well-known clustering methods, such as Ward's method, affinity propagation, and K-means ++. Hum Brain Mapp 38:2643-2665, 2017. © 2017 Wiley Periodicals, Inc.

Magnetic resonance advection imaging of cerebrovascular pulse dynamics

We analyze the pulsatile signal component of dynamic echo planar imaging data from the brain by modeling the dependence between local temporal and spatial signal variability. The resulting magnetic resonance advection imaging maps depict the location of major arteries. Color direction maps allow for visualization of the direction of blood vessels. The potential significance of magnetic resonance advection imaging maps is demonstrated on a functional magnetic resonance imaging data set of 19 healthy subjects. A comparison with the here introduced pulse coherence maps, in which the echo planar imaging signal is correlated with a cardiac pulse signal, shows that the magnetic resonance advection imaging approach results in a better spatial definition without the need for a pulse reference. In addition, it is shown that magnetic resonance advection imaging velocities can be estimates of pulse wave velocities if certain requirements are met, which are specified. Although for this application magnetic resonance advection imaging velocities are not quantitative estimates of pulse wave velocities, they clearly depict local pulsatile dynamics. Magnetic resonance advection imaging can be applied to existing dynamic echo planar imaging data sets with sufficient spatiotemporal resolution. It is discussed whether magnetic resonance advection imaging might have the potential to evolve into a biomarker for the health of the cerebrovascular system.

The effect of acquisition resolution on orientation decoding from V1 BOLD fMRI at 7T

A decade after it was shown that the orientation of visual grating stimuli can be decoded from human visual cortex activity by means of multivariate pattern classification of BOLD fMRI data, numerous studies have investigated which aspects of neuronal activity are reflected in BOLD response patterns and are accessible for decoding. However, it remains inconclusive what the effect of acquisition resolution on BOLD fMRI decoding analyses is. The present study is the first to provide empirical ultra high-field fMRI data recorded at four spatial resolutions (0.8mm, 1.4mm, 2mm, and 3mm isotropic voxel size) on this topic - in order to test hypotheses on the strength and spatial scale of orientation discriminating signals. We present detailed analysis, in line with predictions from previous simulation studies, about how the performance of orientation decoding varies with different acquisition resolutions. Moreover, we also examine different spatial filtering procedures and its effects on orientation decoding. Here we show that higher-resolution scans with subsequent down-sampling or low-pass filtering yield no benefit over scans natively recorded in the corresponding lower resolution regarding decoding accuracy. The orientation-related signal in the BOLD fMRI data is spatially broadband in nature, includes both high spatial frequency components, as well as large-scale biases previously proposed in the literature. Moreover, we found above chance-level contribution from large draining veins to orientation decoding. Acquired raw data were publicly released to facilitate further investigation.

Lies, irony, and contradiction - an annotation of semantic conflict in the movie "Forrest Gump"

Here we extend the information on the structure of the core stimulus of the studyforrest project (http://studyforrest.org) with a description of semantic conflict in the “Forrest Gump” movie. Three observers annotated the movie independently regarding episodes with portrayal of lies, irony or sarcasm. We present frequency statistics, and inter-observer reliability measures that qualify and quantify semantic conflict in the stimulus. While the number of identified events is limited, this annotation nevertheless enriches the knowledge about the complex high-level structure of this stimulus, and can help to evaluate its utility for future studies, and the usability of the existing brain imaging data regarding this aspect of cognition.

A studyforrest extension, simultaneous fMRI and eye gaze recordings during prolonged natural stimulation

Here we present an update of the studyforrest (http://studyforrest.org) dataset that complements the previously released functional magnetic resonance imaging (fMRI) data for natural language processing with a new two-hour 3 Tesla fMRI acquisition while 15 of the original participants were shown an audio-visual version of the stimulus motion picture. We demonstrate with two validation analyses that these new data support modeling specific properties of the complex natural stimulus, as well as a substantial within-subject BOLD response congruency in brain areas related to the processing of auditory inputs, speech, and narrative when compared to the existing fMRI data for audio-only stimulation. In addition, we provide participants' eye gaze location as recorded simultaneously with fMRI, and an additional sample of 15 control participants whose eye gaze trajectories for the entire movie were recorded in a lab setting—to enable studies on attentional processes and comparative investigations on the potential impact of the stimulation setting on these processes.

A studyforrest extension, retinotopic mapping and localization of higher visual areas

The studyforrest (http://studyforrest.org) dataset is likely the largest neuroimaging dataset on natural language and story processing publicly available today. In this article, along with a companion publication, we present an update of this dataset that extends its scope to vision and multi-sensory research. 15 participants of the original cohort volunteered for a series of additional studies: a clinical examination of visual function, a standard retinotopic mapping procedure, and a localization of higher visual areas-such as the fusiform face area. The combination of this update, the previous data releases for the dataset, and the companion publication, which includes neuroimaging and eye tracking data from natural stimulation with a motion picture, form an extremely versatile and comprehensive resource for brain imaging research-with almost six hours of functional neuroimaging data across five different stimulation paradigms for each participant. Furthermore, we describe employed paradigms and present results that document the quality of the data for the purpose of characterising major properties of participants' visual processing stream.

An annotation of cuts, depicted locations, and temporal progression in the motion picture "Forrest Gump"

Here we present an annotation of locations and temporal progression depicted in the movie “Forrest Gump”, as an addition to a large public functional brain imaging dataset ( http://studyforrest.org). The annotation provides information about the exact timing of each of the 870 shots, and the depicted location after every cut with a high, medium, and low level of abstraction. Additionally, four classes are used to distinguish the differences of the depicted time between shots. Each shot is also annotated regarding the type of location (interior/exterior) and time of day. This annotation enables further studies of visual perception, memory of locations, and the perception of time under conditions of real-life complexity using the studyforrest dataset.

Data Citation in Neuroimaging: Proposed Best Practices for Data Identification and Attribution

Data sharing and reuse, while widely accepted as good ideas, have been slow to catch on in any concrete and consistent way. One major hurdle within the scientific community has been the lack of widely accepted standards for citing that data, making it difficult to track usage and measure impact. Within the neuroimaging community, there is a need for a way to not only clearly identify and cite datasets, but also to derive new aggregate sets from multiple sources while clearly maintaining lines of attribution. This work presents a functional prototype of a system to integrate Digital Object Identifiers (DOI) and a standardized metadata schema into a XNAT-based repository workflow, allowing for identification of data at both the project and image level. These item and source level identifiers allow any newly defined combination of images, from any number of projects, to be tagged with a new group-level DOI that automatically inherits the individual attributes and provenance information of its constituent parts. This system enables the tracking of data reuse down to the level of individual images. The implementation of this type of data identification system would impact researchers and data creators, data hosting facilities, and data publishers, but the benefit of having widely accepted standards for data identification and attribution would go far toward making data citation practical and advantageous.

Brain-heart interactions: challenges and opportunities with functional magnetic resonance imaging at ultra-high field

Magnetic resonance imaging (MRI) at ultra-high field (UHF) strengths (7 T and above) offers unique opportunities for studying the human brain with increased spatial resolution, contrast and sensitivity. However, its reliability can be compromised by factors such as head motion, image distortion and non-neural fluctuations of the functional MRI signal. The objective of this review is to provide a critical discussion of the advantages and trade-offs associated with UHF imaging, focusing on the application to studying brain-heart interactions. We describe how UHF MRI may provide contrast and resolution benefits for measuring neural activity of regions involved in the control and mediation of autonomic processes, and in delineating such regions based on anatomical MRI contrast. Limitations arising from confounding signals are discussed, including challenges with distinguishing non-neural physiological effects from the neural signals of interest that reflect cardiorespiratory function. We also consider how recently developed data analysis techniques may be applied to high-field imaging data to uncover novel information about brain-heart interactions.

Building a Science of Individual Differences from fMRI

To date, fMRI research has been concerned primarily with evincing generic principles of brain function through averaging data from multiple subjects. Given rapid developments in both hardware and analysis tools, the field is now poised to study fMRI-derived measures in individual subjects, and to relate these to psychological traits or genetic variations. We discuss issues of validity, reliability and statistical assessment that arise when the focus shifts to individual subjects and that are applicable also to other imaging modalities. We emphasize that individual assessment of neural function with fMRI presents specific challenges and necessitates careful consideration of anatomical and vascular between-subject variability as well as sources of within-subject variability.

Signal prediction by anticipatory relaxation dynamics

Real-time prediction of signals is a task often encountered in control problems as well as by living systems. Here, a parsimonious prediction approach based on the coupling of a linear relaxation-delay system to a smooth, stationary signal is described. The resulting anticipatory relaxation dynamics (ARD) is a frequency-dependent predictor of future signal values. ARD not only approximately predicts signals on average but can anticipate the occurrence of signal peaks, too. This can be explained by recognizing ARD as an input-output system with negative group delay. It is characterized, including its prediction horizon, by its analytically given frequency response function.

The Detection of Emerging Trends Using Wikipedia Traffic Data and Context Networks

Can online media predict new and emerging trends, since there is a relationship between trends in society and their representation in online systems? While several recent studies have used Google Trends as the leading online information source to answer corresponding research questions, we focus on the online encyclopedia Wikipedia often used for deeper topical reading. Wikipedia grants open access to all traffic data and provides lots of additional (semantic) information in a context network besides single keywords. Specifically, we suggest and study context-normalized and time-dependent measures for a topic’s importance based on page-view time series of Wikipedia articles in different languages and articles related to them by internal links. As an example, we present a study of the recently emerging Big Data market with a focus on the Hadoop ecosystem, and compare the capabilities of Wikipedia versus Google in predicting its popularity and life cycles. To support further applications, we have developed an open web platform to share results of Wikipedia analytics, providing context-rich and language-independent relevance measures for emerging trends.

The integration of the internal and external milieu in the insula during dynamic emotional experiences

Whilst external events trigger emotional responses, interoception (the perception of internal physiological states) is fundamental to core emotional experience. By combining high resolution functional neuroimaging with concurrent physiological recordings, we investigated the neural mechanisms of interoceptive integration during free listening to an emotionally salient audio film. We found that cardiac activity, a key interoceptive signal, was robustly synchronised across participants and centrally represented in the posterior insula. Effective connectivity analysis revealed that the anterior insula, specifically tuned to the emotionally salient moments of the audio stream, serves as an integration hub of interoceptive processing: interoceptive states represented in the posterior insula are integrated with exteroceptive representations by the anterior insula to highlight these emotionally salient moments. Our study for the first time demonstrates the insular hierarchy for interoceptive processing during natural emotional experience. These findings provide an ecologically-valid framework for elucidating the neural underpinnings of emotional deficits in neuropsychiatric disorders.

Highest Resolution In Vivo Human Brain MRI Using Prospective Motion Correction

High field MRI systems, such as 7 Tesla (T) scanners, can deliver higher signal to noise ratio (SNR) than lower field scanners and thus allow for the acquisition of data with higher spatial resolution, which is often demanded by users in the fields of clinical and neuroscientific imaging. However, high resolution scans may require long acquisition times, which in turn increase the discomfort for the subject and the risk of subject motion. Even with a cooperative and trained subject, involuntary motion due to heartbeat, swallowing, respiration and changes in muscle tone can cause image artifacts that reduce the effective resolution. In addition, scanning with higher resolution leads to increased sensitivity to even very small movements. Prospective motion correction (PMC) at 3T and 7T has proven to increase image quality in case of subject motion. Although the application of prospective motion correction is becoming more popular, previous articles focused on proof of concept studies and technical descriptions, whereas this paper briefly describes the technical aspects of the optical tracking system, marker fixation and cross calibration and focuses on the application of PMC to very high resolution imaging without intentional motion. In this study we acquired in vivo MR images at 7T using prospective motion correction during long acquisitions. As a result, we present images among the highest, if not the highest resolution of in vivo human brain MRI ever acquired.

A communication hub for a decentralized collaboration on studying real-life cognition

Studying the brain’s behavior in situations of real-life complexity is crucial for an understanding of brain function as a whole. However, methodological difficulties and a general lack of public resources are hindering scientific progress in this domain. This channel will serve as a communication hub to collect relevant resources and curate knowledge about working paradigms, available resources, and analysis techniques.

Portrayed emotions in the movie "Forrest Gump"

Here we present a dataset with a description of portrayed emotions in the movie ”Forrest Gump”. A total of 12 observers independently annotated emotional episodes regarding their temporal location and duration. The nature of an emotion was characterized with basic attributes, such as arousal and valence, as well as explicit emotion category labels. In addition, annotations include a record of the perceptual evidence for the presence of an emotion. Two variants of the movie were annotated separately: 1) an audio-movie version of Forrest Gump that has been used as a stimulus for the acquisition of a large public functional brain imaging dataset, and 2) the original audio-visual movie. We present reliability and consistency estimates that suggest that both stimuli can be used to study visual and auditory emotion cue processing in real-life like situations. Raw annotations from all observers are publicly released in full in order to maximize their utility for a wide range of applications and possible future extensions. In addition, aggregate time series of inter-observer agreement with respect to particular attributes of portrayed emotions are provided to facilitate adoption of these data.

Improving functional magnetic resonance imaging reproducibility

BACKGROUND

The ability to replicate an entire experiment is crucial to the scientific method. With the development of more and more complex paradigms, and the variety of analysis techniques available, fMRI studies are becoming harder to reproduce.

RESULTS

In this article, we aim to provide practical advice to fMRI researchers not versed in computing, in order to make studies more reproducible. All of these steps require researchers to move towards a more open science, in which all aspects of the experimental method are documented and shared.

CONCLUSION

Only by sharing experiments, data, metadata, derived data and analysis workflows will neuroimaging establish itself as a true data science.

A high resolution 7-Tesla resting-state fMRI test-retest dataset with cognitive and physiological measures

Here we present a test-retest dataset of functional magnetic resonance imaging (fMRI) data acquired at rest. 22 participants were scanned during two sessions spaced one week apart. Each session includes two 1.5 mm isotropic whole-brain scans and one 0.75 mm isotropic scan of the prefrontal cortex, giving a total of six time-points. Additionally, the dataset includes measures of mood, sustained attention, blood pressure, respiration, pulse, and the content of self-generated thoughts (mind wandering). This data enables the investigation of sources of both intra- and inter-session variability not only limited to physiological changes, but also including alterations in cognitive and affective states, at high spatial resolution. The dataset is accompanied by a detailed experimental protocol and source code of all stimuli used.