DataViewer3D: An Open-Source, Cross-Platform Multi-Modal Neuroimaging Data Visualization Tool

Project, toolkit, and database of neuroinformatics ecosystem: A summary of previous studies on "Frontiers in Neuroinformatics"

In the field of neuroscience, the core of the cohort study project consists of collection, analysis, and sharing of multi-modal data. Recent years have witnessed a host of efficient and high-quality toolkits published and employed to improve the quality of multi-modal data in the cohort study. In turn, gleaning answers to relevant questions from such a conglomeration of studies is a time-consuming task for cohort researchers. As part of our efforts to tackle this problem, we propose a hierarchical neuroscience knowledge base that consists of projects/organizations, multi-modal databases, and toolkits, so as to facilitate researchers' answer searching process. We first classified studies conducted for the topic "Frontiers in Neuroinformatics" according to the multi-modal data life cycle, and from these studies, information objects as projects/organizations, multi-modal databases, and toolkits have been extracted. Then, we map these information objects into our proposed knowledge base framework. A Python-based query tool has also been developed in tandem for quicker access to the knowledge base, (accessible at https://github.com/Romantic-Pumpkin/PDT_fninf). Finally, based on the constructed knowledge base, we discussed some key research issues and underlying trends in different stages of the multi-modal data life cycle.

NUTMEG: Open Source Software for M/EEG Source Reconstruction

Neurodynamic Utility Toolbox for Magnetoencephalo- and Electroencephalography (NUTMEG) is an open-source MATLAB-based toolbox for the analysis and reconstruction of magnetoencephalography/electroencephalography data in source space. NUTMEG includes a variety of options for the user in data import, preprocessing, source reconstruction, and functional connectivity. A group analysis toolbox allows the user to run a variety of inferential statistics on their data in an easy-to-use GUI-driven format. Importantly, NUTMEG features an interactive five-dimensional data visualization platform. A key feature of NUTMEG is the availability of a large menu of interference cancelation and source reconstruction algorithms. Each NUTMEG operation acts as a stand-alone MATLAB function, allowing the package to be easily adaptable and scripted for the more advanced user for interoperability with other software toolboxes. Therefore, NUTMEG enables a wide range of users access to a complete "sensor-to- source-statistics" analysis pipeline.

Charting the effects of TMS with fMRI: Modulation of cortical recruitment within the distributed network supporting semantic control

Semantic memory comprises our knowledge of the meanings of words and objects but only some of this knowledge is relevant at any given time. Thus, semantic control processes are needed to focus retrieval on relevant information. Research on the neural basis of semantic control has strongly implicated left inferior frontal gyrus (LIFG) but recent work suggests that a wider network supports semantic control, including left posterior middle temporal gyrus (pMTG), right inferior frontal gyrus (RIFG) and pre-supplementary motor area (pre-SMA). In the current study, we used repetitive transcranial magnetic stimulation (1 Hz offline TMS) over LIFG, immediately followed by fMRI, to examine modulation of the semantic network. We compared the effect of stimulation on judgements about strongly-associated words (dog-bone) and weaker associations (dog-beach), since previous studies have found that dominant links can be recovered largely automatically with little engagement of LIFG, while more distant connections require greater control. Even though behavioural performance was maintained in response to TMS, LIFG stimulation increased the effect of semantic control demands in pMTG and pre-SMA, relative to stimulation of a control site (occipital pole). These changes were accompanied by reduced recruitment of both the stimulated region (LIFG) and its right hemisphere homologue (RIFG), particularly for strong associations with low control requirements. Thus repetitive TMS to LIFG modulated the contribution of distributed regions to semantic judgements in two distinct ways.

•

Offline rTMS was used to modulate the semantic control system.

•

fMRI revealed post-stimulation changes in other areas of the semantic control system.

•

Semantic retrieval requires the flexible activation of representations shaped by control processes.

A Review on the Bioinformatics Tools for Neuroimaging

Neuroimaging is a new technique used to create images of the structure and function of the nervous system in the human brain. Currently, it is crucial in scientific fields. Neuroimaging data are becoming of more interest among the circle of neuroimaging experts. Therefore, it is necessary to develop a large amount of neuroimaging tools. This paper gives an overview of the tools that have been used to image the structure and function of the nervous system. This information can help developers, experts, and users gain insight and a better understanding of the neuroimaging tools available, enabling better decision making in choosing tools of particular research interest. Sources, links, and descriptions of the application of each tool are provided in this paper as well. Lastly, this paper presents the language implemented, system requirements, strengths, and weaknesses of the tools that have been widely used to image the structure and function of the nervous system.

Shared neural processes support semantic control and action understanding

Executive-semantic control and action understanding appear to recruit overlapping brain regions but existing evidence from neuroimaging meta-analyses and neuropsychology lacks spatial precision; we therefore manipulated difficulty and feature type (visual vs. action) in a single fMRI study. Harder judgements recruited an executive-semantic network encompassing medial and inferior frontal regions (including LIFG) and posterior temporal cortex (including pMTG). These regions partially overlapped with brain areas involved in action but not visual judgements. In LIFG, the peak responses to action and difficulty were spatially identical across participants, while these responses were overlapping yet spatially distinct in posterior temporal cortex. We propose that the co-activation of LIFG and pMTG allows the flexible retrieval of semantic information, appropriate to the current context; this might be necessary both for semantic control and understanding actions. Feature selection in difficult trials also recruited ventral occipital-temporal areas, not implicated in action understanding.

The connectome of the basal ganglia

The basal ganglia of the laboratory rat consist of a few core regions that are specifically interconnected by efferents and afferents of the central nervous system. In nearly 800 reports of tract-tracing investigations the connectivity of the basal ganglia is documented. The readout of connectivity data and the collation of all the connections of these reports in a database allows to generate a connectome. The collation, curation and analysis of such a huge amount of connectivity data is a great challenge and has not been performed before (Bohland et al. PloS One 4:e7200, 2009) in large connectomics projects based on meta-analysis of tract-tracing studies. Here, the basal ganglia connectome of the rat has been generated and analyzed using the consistent cross-platform and generic framework neuroVIISAS. Several advances of this connectome meta-study have been made: the collation of laterality data, the network-analysis of connectivity strengths and the assignment of regions to a hierarchically organized terminology. The basal ganglia connectome offers differences in contralateral connectivity of motoric regions in contrast to other regions. A modularity analysis of the weighted and directed connectome produced a specific grouping of regions. This result indicates a correlation of structural and functional subsystems. As a new finding, significant reciprocal connections of specific network motifs in this connectome were detected. All three principal basal ganglia pathways (direct, indirect, hyperdirect) could be determined in the connectome. By identifying these pathways it was found that there exist many further equivalent pathways possessing the same length and mean connectivity weight as the principal pathways. Based on the connectome data it is unknown why an excitation pattern may prefer principal rather than other equivalent pathways. In addition to these new findings the local graph-theoretical features of regions of the connectome have been determined. By performing graph theoretical analyses it turns out that beside the caudate putamen further regions like the mesencephalic reticular formation, amygdaloid complex and ventral tegmental area are important nodes in the basal ganglia connectome. The connectome data of this meta-study of tract-tracing reports of the basal ganglia are available for further network studies, the integration into neocortical connectomes and further extensive investigations of the basal ganglia dynamics in population simulations.

On the role of suppression in spatial attention: evidence from negative BOLD in human subcortical and cortical structures

There is clear evidence that spatial attention increases neural responses to attended stimuli in extrastriate visual areas and, to a lesser degree, in earlier visual areas. Other evidence shows that neurons representing unattended locations can also be suppressed. However, the extent to which enhancement and suppression is observed, their stimulus dependence, and the stages of the visual system at which they are expressed remains poorly understood. Using fMRI we set out to characterize both the task and stimulus dependence of neural responses in the lateral geniculate nucleus (LGN), primary visual cortex (V1), and visual motion area (V5) in humans to determine where suppressive and facilitatory effects of spatial attention are expressed. Subjects viewed a lateralized drifting grating stimulus, presented at multiple stimulus contrasts, and performed one of three tasks designed to alter the spatial location of their attention. In retinotopic representations of the stimulus location, we observed increasing attention-dependent facilitation and decreasing dependence on stimulus contrast moving up the visual hierarchy from the LGN to V5. However, in the representations of unattended locations of the LGN and V1, we observed suppression, which was not significantly dependent on the attended stimulus contrast. These suppressive effects were also found in the pulvinar, which has been frequently associated with attention. We provide evidence, therefore, for a spatially selective suppressive mechanism that acts at a subcortical level.

The Selective Role of Premotor Cortex in Speech Perception: A Contribution to Phoneme Judgements but not Speech Comprehension

Several accounts of speech perception propose that the areas involved in producing language are also involved in perceiving it. In line with this view, neuroimaging studies show activation of premotor cortex (PMC) during phoneme judgment tasks; however, there is debate about whether speech perception necessarily involves motor processes, across all task contexts, or whether the contribution of PMC is restricted to tasks requiring explicit phoneme awareness. Some aspects of speech processing, such as mapping sounds onto meaning, may proceed without the involvement of motor speech areas if PMC specifically contributes to the manipulation and categorical perception of phonemes. We applied TMS to three sites—PMC, posterior superior temporal gyrus, and occipital pole—and for the first time within the TMS literature, directly contrasted two speech perception tasks that required explicit phoneme decisions and mapping of speech sounds onto semantic categories, respectively. TMS to PMC disrupted explicit phonological judgments but not access to meaning for the same speech stimuli. TMS to two further sites confirmed that this pattern was site specific and did not reflect a generic difference in the susceptibility of our experimental tasks to TMS: stimulation of pSTG, a site involved in auditory processing, disrupted performance in both language tasks, whereas stimulation of occipital pole had no effect on performance in either task. These findings demonstrate that, although PMC is important for explicit phonological judgments, crucially, PMC is not necessary for mapping speech onto meanings.

Synchronized 2D/3D optical mapping for interactive exploration and real-time visualization of multi-function neurological images

Efficient software with the ability to display multiple neurological image datasets simultaneously with full real-time interactivity is critical for brain disease diagnosis and image-guided planning. In this paper, we describe the creation and function of a new comprehensive software platform that integrates novel algorithms and functions for multiple medical image visualization, processing, and manipulation. We implement an opacity-adjustment algorithm to build 2D lookup tables for multiple slice image display and fusion, which achieves a better visual result than those of using VTK-based methods. We also develop a new real-time 2D and 3D data synchronization scheme for multi-function MR volume and slice image optical mapping and rendering simultaneously through using the same adjustment operation. All these methodologies are integrated into our software framework to provide users with an efficient tool for flexibly, intuitively, and rapidly exploring and analyzing the functional and anatomical MR neurological data. Finally, we validate our new techniques and software platform with visual analysis and task-specific user studies.

Experimental evidence for improved neuroimaging interpretation using three-dimensional graphic models

Three-dimensional (3D) or volumetric visualization is a useful resource for learning about the anatomy of the human brain. However, the effectiveness of 3D spatial visualization has not yet been assessed systematically. This report analyzes whether 3D volumetric visualization helps learners to identify and locate subcortical structures more precisely than classical cross-sectional images based on a two dimensional (2D) approach. Eighty participants were assigned to each experimental condition: 2D cross-sectional visualization vs. 3D volumetric visualization. Both groups were matched for age, gender, visual-spatial ability, and previous knowledge of neuroanatomy. Accuracy in identifying brain structures, execution time, and level of confidence in the response were taken as outcome measures. Moreover, interactive effects between the experimental conditions (2D vs. 3D) and factors such as level of competence (novice vs. expert), image modality (morphological and functional), and difficulty of the structures were analyzed. The percentage of correct answers (hit rate) and level of confidence in responses were significantly higher in the 3D visualization condition than in the 2D. In addition, the response time was significantly lower for the 3D visualization condition in comparison with the 2D. The interaction between the experimental condition (2D vs. 3D) and difficulty was significant, and the 3D condition facilitated the location of difficult images more than the 2D condition. 3D volumetric visualization helps to identify brain structures such as the hippocampus and amygdala, more accurately and rapidly than conventional 2D visualization. This paper discusses the implications of these results with regards to the learning process involved in neuroimaging interpretation.

The connectome viewer toolkit: an open source framework to manage, analyze, and visualize connectomes

Advanced neuroinformatics tools are required for methods of connectome mapping, analysis, and visualization. The inherent multi-modality of connectome datasets poses new challenges for data organization, integration, and sharing. We have designed and implemented the Connectome Viewer Toolkit - a set of free and extensible open source neuroimaging tools written in Python. The key components of the toolkit are as follows: (1) The Connectome File Format is an XML-based container format to standardize multi-modal data integration and structured metadata annotation. (2) The Connectome File Format Library enables management and sharing of connectome files. (3) The Connectome Viewer is an integrated research and development environment for visualization and analysis of multi-modal connectome data. The Connectome Viewer's plugin architecture supports extensions with network analysis packages and an interactive scripting shell, to enable easy development and community contributions. Integration with tools from the scientific Python community allows the leveraging of numerous existing libraries for powerful connectome data mining, exploration, and comparison. We demonstrate the applicability of the Connectome Viewer Toolkit using Diffusion MRI datasets processed by the Connectome Mapper. The Connectome Viewer Toolkit is available from http://www.cmtk.org/

Functional asymmetries in the representation of noise-vocoded speech

It is generally accepted that, while speech is processed bilaterally in auditory cortical areas, complementary analyses of the speech signal are carried out across the hemispheres. However, the Asymmetric Sampling in Time (AST) model (Poeppel, 2003) suggests that there is functional asymmetry due to different time scales of temporal integration in each hemisphere. The right hemisphere preferentially processes slow modulations commensurate with the theta frequency band (~4-8 Hz), whereas the left hemisphere is more sensitive to fast temporal modulations in the gamma frequency range (~25-50 Hz). Here we examined the perception of noise-vocoded, i.e. spectrally-degraded, words. Magnetoencephalography (MEG) beamformer analyses were used to determine where and how noise-vocoded speech is represented in terms of changes in power resulting from neuronal activity. The outputs of beamformer spatial filters were used to delineate the temporal dynamics of these changes in power. Beamformer analyses localised low-frequency "delta" (1-4 Hz) and "theta" (3-6 Hz) changes in total power to the left hemisphere and high-frequency "gamma" (60-80 Hz, 80-100 Hz) changes in total power to the right hemisphere. Time-frequency analyses confirmed the frequency content and timing of changes in power in the left and right hemispheres. Together the beamformer and time-frequency analyses demonstrate a functional asymmetry in the representation of noise-vocoded words that is inconsistent with the AST model, at least in brain areas outside of primary auditory cortex.

Heterogeneity of the left temporal lobe in semantic representation and control: priming multiple versus single meanings of ambiguous words

Semantic judgments involve both representations of meaning plus executive mechanisms that guide knowledge retrieval in a task-appropriate way. These 2 components of semantic cognition—representation and control—are commonly linked to left temporal and prefrontal cortex, respectively. This simple proposal, however, remains contentious because in most functional neuroimaging studies to date, the number of concepts being activated and the involvement of executive processes during retrieval are confounded. Using functional magnetic resonance imaging, we examined a task in which semantic representation and control demands were dissociable. Words with multiple meanings like “bank” served as targets in a double-prime paradigm, in which multiple meaning activation and maximal executive demands loaded onto different priming conditions. Anterior inferior temporal gyrus (ITG) was sensitive to the number of meanings that were retrieved, suggesting a role for this region in semantic representation, while posterior middle temporal gyrus (pMTG) and inferior frontal cortex showed greater activation in conditions that maximized executive demands. These results support a functional dissociation between left ITG and pMTG, consistent with a revised neural organization in which left prefrontal and posterior temporal areas work together to underpin aspects of semantic control.

fMRI data visualization with BrainBlend and Blender

The visualization and exploration of neuroimaging data is important for the analysis of anatomical and functional magnetic resonance (MR) images and thresholded statistical parametric maps. While two-dimensional orthogonal views of neuroimaging data are used to display statistical analyses, real three-dimensional (3d) depictions are helpful for showing the spatial distribution of a functional network, as well as its temporal evolution. However, viewers that are freely available on the internet offer only limited rendering capabilities and depictions of temporal changes of the blood oxygen level-dependent (BOLD) response. In this article, we present BrainBlend, a toolbox for the software package Statistical Parametric Mapping (SPM), that generates voxeldata files to be used with the open-source 3d-software "Blender". Our interface between SPM and Blender permits the use of any Analyze- and Nifti-file for the creation of images and animations of transparent volumetric objects. Different kinds of anatomical, functional and statistical data can be rendered as volumetric objects in order to convey an immediate understanding of the three-dimensional shape. Representations of functional networks can be animated using a time course extracted from the general linear model or the independent component analysis. Relative BOLD activations of functional MR-images can be calculated for a time-resolved depiction of hemodynamic changes. The resulting animation can be displayed along with its corresponding paradigm matrix and the presented stimuli. BrainBlend is particularly suitable for the visual exploration of interactions between functional networks, for time-resolved animations of BOLD changes and meets high demands on visual quality in images and animations.