Fusing concurrent EEG-fMRI with dynamic causal modeling: application to effective connectivity during face perception

MCAN: Multimodal Causal Adversarial Networks for Dynamic Effective Connectivity Learning From fMRI and EEG Data

Dynamic effective connectivity (DEC) is the accumulation of effective connectivity in the time dimension, which can describe the continuous neural activities in the brain. Recently, learning DEC from functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) data has attracted the attention of neuroinformatics researchers. However, the current methods fail to consider the gap between the fMRI and EEG modality, which can not precisely learn the DEC network from multimodal data. In this paper, we propose a multimodal causal adversarial network for DEC learning, named MCAN. The MCAN contains two modules: multimodal causal generator and multimodal causal discriminator. First, MCAN employs a multimodal causal generator with an attention-guided layer to produce a posterior signal and output a set of DEC networks. Then, the proposed method uses a multimodal causal discriminator to unsupervised calculate the joint gradient, which directs the update of the whole network. The experimental results on simulated data sets show that MCAN is superior to other state-of-the-art methods in learning the network structure of DEC and can effectively estimate the brain states. The experimental results on real data sets show that MCAN can better reveal abnormal patterns of brain activity and has good application potential in brain network analysis.

Combining electrodermal activity analysis and dynamic causal modeling to investigate the visual-odor multimodal integration during face perception

Objective. This study presents a novel methodological approach for incorporating information related to the peripheral sympathetic response into the investigation of neural dynamics. Particularly, we explore how hedonic contextual olfactory stimuli influence the processing of neutral faces in terms of sympathetic response, event-related potentials and effective connectivity analysis. The objective is to investigate how the emotional valence of odors influences the cortical connectivity underlying face processing and the role of face-induced sympathetic arousal in this visual-olfactory multimodal integration.Approach. To this aim, we combine electrodermal activity (EDA) analysis and dynamic causal modeling to examine changes in cortico-cortical interactions.Results. The results reveal that stimuli arising sympathetic EDA responses are associated with a more negative N170 amplitude, which may be a marker of heightened arousal in response to faces. Hedonic odors, on the other hand, lead to a more negative N1 component and a reduced the vertex positive potential when they are unpleasant or pleasant. Concerning connectivity, unpleasant odors strengthen the forward connection from the inferior temporal gyrus (ITG) to the middle temporal gyrus, which is involved in processing changeable facial features. Conversely, the occurrence of sympathetic responses after a stimulus is correlated with an inhibition of this same connection and an enhancement of the backward connection from ITG to the fusiform face gyrus.Significance. These findings suggest that unpleasant odors may enhance the interpretation of emotional expressions and mental states, while faces capable of eliciting sympathetic arousal prioritize identity processing.

Multimodal investigations of emotional face processing and social trait judgment of faces

Faces are among the most important visual stimuli that humans perceive in everyday life. While extensive literature has examined emotional processing and social evaluations of faces, most studies have examined either topic using unimodal approaches. In this review, we promote the use of multimodal cognitive neuroscience approaches to study these processes, using two lines of research as examples: ambiguity in facial expressions of emotion and social trait judgment of faces. In the first set of studies, we identified an event-related potential that signals emotion ambiguity using electroencephalography and we found convergent neural responses to emotion ambiguity using functional neuroimaging and single-neuron recordings. In the second set of studies, we discuss how different neuroimaging and personality-dimensional approaches together provide new insights into social trait judgments of faces. In both sets of studies, we provide an in-depth comparison between neurotypicals and people with autism spectrum disorder. We offer a computational account for the behavioral and neural markers of the different facial processing between the two groups. Finally, we suggest new practices for studying the emotional processing and social evaluations of faces. All data discussed in the case studies of this review are publicly available.

Using multi-modal neuroimaging to characterise social brain specialisation in infants

The specialised regional functionality of the mature human cortex partly emerges through experience-dependent specialisation during early development. Our existing understanding of functional specialisation in the infant brain is based on evidence from unitary imaging modalities and has thus focused on isolated estimates of spatial or temporal selectivity of neural or haemodynamic activation, giving an incomplete picture. We speculate that functional specialisation will be underpinned by better coordinated haemodynamic and metabolic changes in a broadly orchestrated physiological response. To enable researchers to track this process through development, we develop new tools that allow the simultaneous measurement of coordinated neural activity (EEG), metabolic rate, and oxygenated blood supply (broadband near-infrared spectroscopy) in the awake infant. In 4- to 7-month-old infants, we use these new tools to show that social processing is accompanied by spatially and temporally specific increases in coupled activation in the temporal-parietal junction, a core hub region of the adult social brain. During non-social processing, coupled activation decreased in the same region, indicating specificity to social processing. Coupling was strongest with high-frequency brain activity (beta and gamma), consistent with the greater energetic requirements and more localised action of high-frequency brain activity. The development of simultaneous multimodal neural measures will enable future researchers to open new vistas in understanding functional specialisation of the brain.

Functional Segregation within the Dorsal Frontoparietal Network: A Multimodal Dynamic Causal Modeling Study

Discrimination and integration of motion direction requires the interplay of multiple brain areas. Theoretical accounts of perception suggest that stimulus-related (i.e., exogenous) and decision-related (i.e., endogenous) factors affect distributed neuronal processing at different levels of the visual hierarchy. To test these predictions, we measured brain activity of healthy participants during a motion discrimination task, using electroencephalography (EEG) and functional magnetic resonance imaging (fMRI). We independently modeled the impact of exogenous factors (task demand) and endogenous factors (perceptual decision-making) on the activity of the motion discrimination network and applied Dynamic Causal Modeling (DCM) to both modalities. DCM for event-related potentials (DCM-ERP) revealed that task demand impacted the reciprocal connections between the primary visual cortex (V1) and medial temporal areas (V5). With practice, higher visual areas were increasingly involved, as revealed by DCM-fMRI. Perceptual decision-making modulated higher levels (e.g., V5-to-Frontal Eye Fields, FEF), in a manner predictive of performance. Our data suggest that lower levels of the visual network support early, feature-based selection of responses, especially when learning strategies have not been implemented. In contrast, perceptual decision-making operates at higher levels of the visual hierarchy by integrating sensory information with the internal state of the subject.

Modeling temporal dynamics of face processing in youth and adults

A hierarchical model of temporal dynamics was examined in adults (n = 34) and youth (n = 46) across the stages of face processing during the perception of static and dynamic faces. Three ERP components (P100, N170, N250) and spectral power in the mu range were extracted, corresponding to cognitive stages of face processing: low-level vision processing, structural encoding, higher-order processing, and action understanding. Youth and adults exhibited similar yet distinct patterns of hierarchical temporal dynamics such that earlier cognitive stages predicted later stages, directly and indirectly. However, latent factors indicated unique profiles related to behavioral performance for adults and youth and age as a continuous factor. The application of path analysis to electrophysiological data can yield novel insights into the cortical dynamics of social information processing.

When Is Simultaneous Recording Necessary? A Guide for Researchers Considering Combined EEG-fMRI

Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) provide non-invasive measures of brain activity at varying spatial and temporal scales, offering different views on brain function for both clinical and experimental applications. Simultaneous recording of these measures attempts to maximize the respective strengths of each method, while compensating for their weaknesses. However, combined recording is not necessary to address all research questions of interest, and experiments may have greater statistical power to detect effects by maximizing the signal-to-noise ratio in separate recording sessions. While several existing papers discuss the reasons for or against combined recording, this article aims to synthesize these arguments into a flow chart of questions that researchers can consider when deciding whether to record EEG and fMRI separately or simultaneously. Given the potential advantages of simultaneous EEG-fMRI, the aim is to provide an initial overview of the most important concepts and to direct readers to relevant literature that will aid them in this decision.

Sensing and seeing associated with overlapping occipitoparietal activation in simultaneous EEG-fMRI

The presence of a change in a visual scene can influence brain activity and behavior, even in the absence of full conscious report. It may be possible for us to sense that such a change has occurred, even if we cannot specify exactly where or what it was. Despite existing evidence from electroencephalogram (EEG) and eye-tracking data, it is still unclear how this partial level of awareness relates to functional magnetic resonance imaging (fMRI) blood oxygen level dependent (BOLD) activation. Using EEG, fMRI, and a change blindness paradigm, we found multi-modal evidence to suggest that sensing a change is distinguishable from being blind to it. Specifically, trials during which participants could detect the presence of a colour change but not identify the location of the change (sense trials), were compared to those where participants could both detect and localise the change (localise or see trials), as well as change blind trials. In EEG, late parietal positivity and N2 amplitudes were larger for localised changes only, when compared to change blindness. However, ERP-informed fMRI analysis found no voxels with activation that significantly co-varied with fluctuations in single-trial late positivity amplitudes. In fMRI, a range of visual (BA17,18), parietal (BA7,40), and mid-brain (anterior cingulate, BA24) areas showed increased fMRI BOLD activation when a change was sensed, compared to change blindness. These visual and parietal areas are commonly implicated as the storage sites of visual working memory, and we therefore argue that sensing may not be explained by a lack of stored representation of the visual display. Both seeing and sensing a change were associated with an overlapping occipitoparietal network of activation when compared to blind trials, suggesting that the quality of the visual representation, rather than the lack of one, may result in partial awareness during the change blindness paradigm.

Robust Autoregression with Exogenous Input Model for System Identification and Predicting

Autoregression with exogenous input (ARX) is a widely used model to estimate the dynamic relationships between neurophysiological signals and other physiological parameters. Nevertheless, biological signals, such as electroencephalogram (EEG), arterial blood pressure (ABP), and intracranial pressure (ICP), are inevitably contaminated by unexpected artifacts, which may distort the parameter estimation due to the use of the L2 norm structure. In this paper, we defined the ARX in the Lp (p ≤ 1) norm space with the aim of resisting outlier influence and designed a feasible iteration procedure to estimate model parameters. A quantitative evaluation with various outlier conditions demonstrated that the proposed method could estimate ARX parameters more robustly than conventional methods. Testing with the resting-state EEG with ocular artifacts demonstrated that the proposed method could predict missing data with less influence from the artifacts. In addition, the results on ICP and ABP data further verified its efficiency for model fitting and system identification. The proposed Lp-ARX may help capture system parameters reliably with various input and output signals that are contaminated with artifacts.

Event-related network changes unfold the dynamics of cortical integration during face processing

Face perception arises from a collective activation of brain regions in the occipital, parietal and temporal cortices. Despite the wide acknowledgment that these regions act in an intertwined network, the network behavior itself is poorly understood. Here we present a study in which time-varying connectivity estimated from EEG activity elicited by facial expressions presentation was characterized using graph-theoretical measures of node centrality and global network topology. Results revealed that face perception results from a dynamic reshaping of the network architecture, characterized by the emergence of hubs located in the occipital and temporal regions of the scalp. The importance of these nodes can be observed from the early stages of visual processing and reaches a climax in the same time-window in which the face-sensitive N170 is observed. Furthermore, using Granger causality, we found that the time-evolving centrality of these nodes is associated with ERP amplitude, providing a direct link between the network state and local neural response. Additionally, investigating global network topology by means of small-worldness and modularity, we found that face processing requires a functional network with a strong small-world organization that maximizes integration, at the cost of segregated subdivisions. Interestingly, we found that this architecture is not static, but instead, it is implemented by the network from stimulus onset to ~200 ms. Altogether, this study reveals the event-related changes underlying face processing at the network level, suggesting that a distributed processing mechanism operates through dynamically weighting the contribution of the cortical regions involved.

Face Recognition Brain Functional Connectivity in Patients With Major Depression: A Brain Source Localization Study by ERP

Objective: Patients with major depressive disorder (MDD) presents with face recognition defects. These defects negatively affect their social interactions. However, the cause of these defects is not clear. This study sought to explore whether MDD patients develop facial perceptual processing disorders with characteristics of brain functional connectivity (FC). Methods: Event-related potential (ERP) was used to explore differences between 20 MDD patients and 20 healthy participants with face and non-face recognition tasks based on 64 EEG parameters. After pre-processing of EEG data and source reconstruction using the minimum-norm estimate (MNE), data were converted to AAL90 template to obtain a time series of 90 brain regions. EEG power spectra were determined using Fieldtrip incorporating a Fast Fourier transform. FC was determined for all pairs of brain signals for theta band using debiased estimate of weighted phase-lag index (wPLI) in Fieldtrip. To explore group differences in wPLI, independent t-tests were performed with p < 0.05 to indicate statistical significance. False discovery rate (FDR) correction was used to adjust p-values. Results: The findings showed that amplitude induction by face pictures was higher compared with that of non-face pictures both in MDD and healthy control (HC) groups. Face recognition amplitude in MDD group was lower compared with that in the HC group. Two time periods with significant differences were then selected for further analysis. Analysis showed that FC was stronger in the MDD group compared with that in the HC group in most brain regions in both periods. However, only one FC between two brain regions in HC group was stronger compared with that in the MDD group. Conclusion: Dysfunction in brain FC among MDD patients is a relatively complex phenomenon, exhibiting stronger and multiple connectivity with several brain regions of emotions. The findings of the current study indicate that the brain FC of MDD patients is more complex and less efficient in the initial stage of face recognition.

Co-localization of theta-band activity and hemodynamic responses during face perception: simultaneous electroencephalography and functional near-infrared spectroscopy recordings

Face-specific neural processes in the human brain have been localized to multiple anatomical structures and associated with diverse and dynamic social functions. The question of how various face-related systems and functions may be bound together remains an active area of investigation. We hypothesize that face processing may be associated with specific frequency band oscillations that serve to integrate distributed face processing systems. Using a multimodal imaging approach, including electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS), simultaneous signals were acquired during face and object picture viewing. As expected for face processing, hemodynamic activity in the right occipital face area (OFA) increased during face viewing compared to object viewing, and in a subset of participants, the expected N170 EEG response was observed for faces. Based on recently reported associations between the theta band and visual processing, we hypothesized that increased hemodynamic activity in a face processing area would also be associated with greater theta-band activity originating in the same area. Consistent with our hypothesis, theta-band oscillations were also localized to the right OFA for faces, whereas alpha- and beta-band oscillations were not. Together, these findings suggest that theta-band oscillations originating in the OFA may be part of the distributed face-specific processing mechanism.

Towards understanding neural network signatures of motor skill learning in Parkinson's disease and healthy aging

In the past decade, neurorehabilitation has been shown to be an effective therapeutic supplement for patients with Parkinson's disease (PD). However, patients still experience severe problems with the consolidation of learned motor skills. Knowledge on the neural correlates underlying this process is thus essential to optimize rehabilitation for PD. This review investigates the existing studies on neural network connectivity changes in relation to motor learning in healthy aging and PD and critically evaluates the imaging methods used from a methodological point of view. The results indicate that despite neurodegeneration there is still potential to modify connectivity within and between motor and cognitive networks in response to motor training, although these alterations largely bypass the most affected regions in PD. However, so far training-related changes are inferred and possible relationships are not substantiated by brain-behavior correlations. Furthermore, the studies included suffer from many methodological drawbacks. This review also highlights the potential for using neural network measures as predictors for the response to rehabilitation, mainly based on work in young healthy adults. We speculate that future approaches, including graph theory and multimodal neuroimaging, may be more sensitive than brain activation patterns and model-based connectivity maps to capture the effects of motor learning. Overall, this review suggests that methodological developments in neuroimaging will eventually provide more detailed knowledge on how neural networks are modified by training, thereby paving the way for optimized neurorehabilitation for patients.

Consistent Gradient of Performance and Decoding of Stimulus Type and Valence From Local and Network Activity

The individual differences approach focuses on the variation of behavioral and neural signatures across subjects. In this context, we searched for intracranial neural markers of performance in three individuals with distinct behavioral patterns (efficient, borderline, and inefficient) in a dual-valence task assessing facial and lexical emotion recognition. First, we performed a preliminary study to replicate well-established evoked responses in relevant brain regions. Then, we examined time series data and network connectivity, combined with multivariate pattern analyses and machine learning, to explore electrophysiological differences in resting-state versus task-related activity across subjects. Next, using the same methodological approach, we assessed the neural decoding of performance for different dimensions of the task. The classification of time series data mirrored the behavioral gradient across subjects for stimulus type but not for valence. However, network-based measures reflected the subjects' hierarchical profiles for both stimulus types and valence. Therefore, this measure serves as a sensitive marker for capturing distributed processes such as emotional valence discrimination, which relies on an extended set of regions. Network measures combined with classification methods may offer useful insights to study single subjects and understand inter-individual performance variability. Promisingly, this approach could eventually be extrapolated to other neuroscientific techniques.

Evaluating the organizational structure and specificity of network topology within the face processing system

There is increasing appreciation that network-level interactions among regions produce components of face processing previously ascribed to individual regions. Our goals were to use an exhaustive data-driven approach to derive and quantify the topology of directed functional connections within a priori defined nodes of the face processing network and evaluate whether the topology is category-specific. Young adults were scanned with fMRI as they viewed movies of faces, objects, and scenes. We employed GIMME to model effective connectivity among core and extended face processing regions, which allowed us to evaluate all possible directional connections, under each viewing condition (face, object, place). During face processing, we observed directional connections from the right posterior superior temporal sulcus to both the right occipital face area and right fusiform face area (FFA), which does not reflect the topology reported in prior studies. We observed connectivity between core and extended regions during face processing, but this limited to a feed-forward connection from the FFA to the amygdala. Finally, the topology of connections was unique to face processing. These findings suggest that the pattern of directed functional connections within the face processing network, particularly in the right core regions, may not be as hierarchical and feed-forward as described previously. Our findings support the notion that topologies of network connections are specialized, emergent, and dynamically responsive to task demands.

Structural and effective brain connectivity underlying biological motion detection

The perception of actions underwrites a wide range of socio-cognitive functions. Previous neuroimaging and lesion studies identified several components of the brain network for visual biological motion (BM) processing, but interactions among these components and their relationship to behavior remain little understood. Here, using a recently developed integrative analysis of structural and effective connectivity derived from high angular resolution diffusion imaging (HARDI) and functional magnetic resonance imaging (fMRI), we assess the cerebro-cerebellar network for processing of camouflaged point-light BM. Dynamic causal modeling (DCM) informed by probabilistic tractography indicates that the right superior temporal sulcus (STS) serves as an integrator within the temporal module. However, the STS does not appear to be a "gatekeeper" in the functional integration of the occipito-temporal and frontal regions: The fusiform gyrus (FFG) and middle temporal cortex (MTC) are also connected to the right inferior frontal gyrus (IFG) and insula, indicating multiple parallel pathways. BM-specific loops of effective connectivity are seen between the left lateral cerebellar lobule Crus I and right STS, as well as between the left Crus I and right insula. The prevalence of a structural pathway between the FFG and STS is associated with better BM detection. Moreover, a canonical variate analysis shows that the visual sensitivity to BM is best predicted by BM-specific effective connectivity from the FFG to STS and from the IFG, insula, and STS to the early visual cortex. Overall, the study characterizes the architecture of the cerebro-cerebellar network for BM processing and offers prospects for assessing the social brain.

Optimizing Within-Subject Experimental Designs for jICA of Multi-Channel ERP and fMRI

Joint independent component analysis (jICA) can be applied within subject for fusion of multi-channel event-related potentials (ERP) and functional magnetic resonance imaging (fMRI), to measure brain function at high spatiotemporal resolution (Mangalathu-Arumana et al., 2012). However, the impact of experimental design choices on jICA performance has not been systematically studied. Here, the sensitivity of jICA for recovering neural sources in individual data was evaluated as a function of imaging SNR, number of independent representations of the ERP/fMRI data, relationship between instantiations of the joint ERP/fMRI activity (linear, non-linear, uncoupled), and type of sources (varying parametrically and non-parametrically across representations of the data), using computer simulations. Neural sources were simulated with spatiotemporal and noise attributes derived from experimental data. The best performance, maximizing both cross-modal data fusion and the separation of brain sources, occurred with a moderate number of representations of the ERP/fMRI data (10-30), as in a mixed block/event related experimental design. Importantly, the type of relationship between instantiations of the ERP/fMRI activity, whether linear, non-linear or uncoupled, did not in itself impact jICA performance, and was accurately recovered in the common profiles (i.e., mixing coefficients). Thus, jICA provides an unbiased way to characterize the relationship between ERP and fMRI activity across brain regions, in individual data, rendering it potentially useful for characterizing pathological conditions in which neurovascular coupling is adversely affected.

Translational Functional Neuroimaging in the Explanation of Depression

Translation as a notion and procedure is deeply embodied in medical science and education. Translation includes the possibility to translate data across disciplines to improve diagnosis and treatment procedures. The evidence accumulated using translation serves as a vehicle for reification of medical diagnoses. There are promising, established post hoc correlations between the different types of clinical tools (interviews and inventories) and neuroscience. The various measures represent statistical correlations that must now be integrated into diagnostic standards and procedures but this, as a whole, is a step forward towards a better understanding of the mechanisms underlying psychopathology in general and depression in particular. Here, we focus on functional magnetic resonance imaging studies using a trans-disciplinary approach and attempt to establish bridges between the different fields. We will selectively highlight research areas such as imaging genetics, imaging immunology and multimodal imaging, as related to the diagnosis and management of depression.

Interactions between the anterior cingulate-insula network and the fronto-parietal network during perceptual decision-making

Information processing in the human brain during cognitively demanding goal-directed tasks is thought to involve several large-scale brain networks, including the anterior cingulate-insula network (aCIN) and the fronto-parietal network (FPN). Recent functional MRI (fMRI) studies have provided clues that the aCIN initiates activity changes in the FPN. However, when and how often these networks interact remains largely unknown to date. Here, we systematically examined the oscillatory interactions between the aCIN and the FPN by using the spectral Granger causality analysis of reconstructed brain source signals from the scalp electroencephalography (EEG) recorded from human participants performing a face-house perceptual categorization task. We investigated how the aCIN and the FPN interact, what the temporal sequence of events in these nodes is, and what frequency bands of information flow bind these nodes in networks. We found that beta band (13-30Hz) and gamma (30-100Hz) bands of interactions are involved between the aCIN and the FPN during decision-making tasks. In gamma band, the aCIN initiated the Granger causal control over the FPN in 25-225 ms timeframe. In beta band, the FPN achieved a control over the aCIN in 225-425 ms timeframe. These band-specific time-dependent Granger causal controls of the aCIN and the FPN were retained for behaviorally harder decision-making tasks. These findings of times and frequencies of oscillatory interactions in the aCIN and FPN provide us new insights into the general neural mechanisms for sensory information-guided, goal-directed behaviors, including perceptual decision-making processes.

Mental stress assessment using simultaneous measurement of EEG and fNIRS

Previous studies reported mental stress as one of the major contributing factors leading to various diseases such as heart attack, depression and stroke. An accurate stress assessment method may thus be of importance to clinical intervention and disease prevention. We propose a joint independent component analysis (jICA) based approach to fuse simultaneous measurement of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) on the prefrontal cortex (PFC) as a means of stress assessment. For the purpose of this study, stress was induced by using an established mental arithmetic task under time pressure with negative feedback. The induction of mental stress was confirmed by salivary alpha amylase test. Experiment results showed that the proposed fusion of EEG and fNIRS measurements improves the classification accuracy of mental stress by +3.4% compared to EEG alone and +11% compared to fNIRS alone. Similar improvements were also observed in sensitivity and specificity of proposed approach over unimodal EEG/fNIRS. Our study suggests that combination of EEG (frontal alpha rhythm) and fNIRS (concentration change of oxygenated hemoglobin) could be a potential means to assess mental stress objectively.

Face or House Image Perception: Beta and Gamma Bands of Oscillations in Brain Networks Carry Out Decision-Making

Previous functional magnetic resonance imaging studies have consistently shown that perception of visual objects, such as faces and houses, involves distributed brain networks that include the fusiform face area (FFA), parahippocampal place area (PPA), and dorsolateral prefrontal cortex (DLPFC). These regions are commonly observed to be coactivated in BOLD measurements during perception of visual objects. In this study, we aimed to disentangle node-level and network-level activities in millisecond timescale of perception and decision-making in attempts to answer questions about timing and frequency of brain oscillatory activities. We used clear and noisy face-house image categorization tasks and human scalp electroencephalography recordings combined with source reconstruction techniques to study when and how oscillatory activity organizes within the FFA, PPA, and DLPFC. We uncovered the dynamics of two oscillatory networks-beta (13-30 Hz) and gamma (30-100 Hz). In beta band, the node and network activities were enhanced in time frame of 125-250 msec after stimulus onset, the FFA and PPA acted as main outflow hubs and the DLPFC as a main inflow hub, and network activities negatively correlated with behavior measures of noise levels (response times). In gamma band, node and network activities were elevated in time frame of 0-125 msec after stimulus onset, the DLPFC acted as a main outflow hub, and finally network activities were positively correlated with the noise level. These findings broaden our understanding of temporal evolution of node and network features associated with visual perceptual decision-making.

Convergent evidence for hierarchical prediction networks from human electrocorticography and magnetoencephalography

We propose that sensory inputs are processed in terms of optimised predictions and prediction error signals within hierarchical neurocognitive models. The combination of non-invasive brain imaging and generative network models has provided support for hierarchical frontotemporal interactions in oddball tasks, including recent identification of a temporal expectancy signal acting on prefrontal cortex. However, these studies are limited by the need to invert magnetoencephalographic or electroencephalographic sensor signals to localise activity from cortical 'nodes' in the network, or to infer neural responses from indirect measures such as the fMRI BOLD signal. To overcome this limitation, we examined frontotemporal interactions estimated from direct cortical recordings from two human participants with cortical electrode grids (electrocorticography - ECoG). Their frontotemporal network dynamics were compared to those identified by magnetoencephalography (MEG) in forty healthy adults. All participants performed the same auditory oddball task with standard tones interspersed with five deviant tone types. We normalised post-operative electrode locations to standardised anatomic space, to compare across modalities, and inverted the MEG to cortical sources using the estimated lead field from subject-specific head models. A mismatch negativity signal in frontal and temporal cortex was identified in all subjects. Generative models of the electrocorticographic and magnetoencephalographic data were separately compared using the free-energy estimate of the model evidence. Model comparison confirmed the same critical features of hierarchical frontotemporal networks in each patient as in the group-wise MEG analysis. These features included bilateral, feedforward and feedback frontotemporal modulated connectivity, in addition to an asymmetric expectancy driving input on left frontal cortex. The invasive ECoG provides an important step in construct validation of the use of neural generative models of MEG, which in turn enables generalisation to larger populations. Together, they give convergent evidence for the hierarchical interactions in frontotemporal networks for expectation and processing of sensory inputs.

Brain Signals of Face Processing as Revealed by Event-Related Potentials

We analyze the functional significance of different event-related potentials (ERPs) as electrophysiological indices of face perception and face recognition, according to cognitive and neurofunctional models of face processing. Initially, the processing of faces seems to be supported by early extrastriate occipital cortices and revealed by modulations of the occipital P1. This early response is thought to reflect the detection of certain primary structural aspects indicating the presence grosso modo of a face within the visual field. The posterior-temporal N170 is more sensitive to the detection of faces as complex-structured stimuli and, therefore, to the presence of its distinctive organizational characteristics prior to within-category identification. In turn, the relatively late and probably more rostrally generated N250r and N400-like responses might respectively indicate processes of access and retrieval of face-related information, which is stored in long-term memory (LTM). New methods of analysis of electrophysiological and neuroanatomical data, namely, dynamic causal modeling, single-trial and time-frequency analyses, are highly recommended to advance in the knowledge of those brain mechanisms concerning face processing.

Development of effective connectivity in the core network for face perception

This study measured effective connectivity within the core face network in young children using a paediatric magnetoencephalograph (MEG). Dynamic casual modeling (DCM) of brain responses was performed in a group of adults (N = 14) and a group of young children aged from 3 to 6 years (N = 15). Three candidate DCM models were tested, and the fits of the MEG data to the three models were compared at both individual and group levels. The results show that the connectivity structure of the core face network differs significantly between adults and children. Further, the relative strengths of face network connections were differentially modulated by experimental conditions in the two groups. These results support the interpretation that the core face network undergoes significant structural configuration and functional specialization between four years of age and adulthood.

Reciprocal interactions of the SMA and cingulate cortex sustain premovement activity for voluntary actions

Voluntary action is one of the core functions of the human brain, and is accompanied by the well known readiness potential or Bereitschaftspotential. A network of cortical areas is responsible for the motor preparation process, including the anterior mid-cingulate cortex (aMCC) and the SMA. However, the relationship between activity in these regions during movement preparation and the readiness potential is poorly understood. We examined this relationship by integrating simultaneously acquired EEG and fMRI through computational modeling. We first observed that global field power of premovement neural activity showed a specific correlation with BOLD responses in the aMCC. We then used dynamic causal modeling to infer premovement interactions between these regions and their relationship to the premovement neural activity underlying the readiness potential. These analyses suggest that SMA and aMCC have strong reciprocal connections that act to sustain each other's activity, and that this interaction is mediated during movement preparation according to the readiness potential amplitude, as reflected in global cortical field power. Our study suggests that the reciprocal connections between SMA and aMCC are important to maintain the sustained activity of the readiness potential before movement and lead to a weak system instability at movement onset. We suggest that the effective connectivity of this network underlies its functional role in the preparation of self-generated actions.

Electrical stimulation over bilateral occipito-temporal regions reduces N170 in the right hemisphere and the composite face effect

Transcranial direct current stimulation (tDCS) is a non-invasive brain stimulation technique that can modulate cortical excitability. Although the clinical value of tDCS has been advocated, the potential of tDCS in cognitive rehabilitation of face processing deficits is less understood. Face processing has been associated with the occipito-temporal cortex (OT). The present study investigated whether face processing in healthy adults can be modulated by applying tDCS over the OT. Experiment 1 investigated whether tDCS can affect N170, a face-sensitive ERP component, with a face orientation judgment task. The N170 in the right hemisphere was reduced in active stimulation conditions compared with the sham stimulation condition for both upright faces and inverted faces. Experiment 2 further demonstrated that tDCS can modulate the composite face effect, a type of holistic processing that reflects the obligatory attention to all parts of a face. The composite face effect was reduced in active stimulation conditions compared with the sham stimulation condition. Additionally, the current polarity did not modulate the effect of tDCS in the two experiments. The present study demonstrates that N170 can be causally manipulated by stimulating the OT with weak currents. Furthermore, our study provides evidence that obligatory attention to all parts of a face can be affected by the commonly used tDCS parameter setting.

The amygdala's response to face and emotional information and potential category-specific modulation of temporal cortex as a function of emotion

The amygdala has been implicated in the processing of emotion and animacy information and to be responsive to novelty. However, the way in which these functions interact is poorly understood. Subjects (N = 30) viewed threatening or neutral images that could be either animate (facial expressions) or inanimate (objects) in the context of a dot probe task. The amygdala showed responses to both emotional and animacy information, but no emotion by stimulus-type interaction; i.e., emotional face and object stimuli, when matched for arousal and valence, generate comparable amygdala activity relative to neutral face and object stimuli. Additionally, a habituation effect was not seen in amygdala; however, increased amygdala activity was observed for incongruent relative to congruent negative trials in second vs. first exposures. Furthermore, medial fusiform gyrus showed increased response to inanimate stimuli, while superior temporal sulcus showed increased response to animate stimuli. Greater functional connectivity between bilateral amygdala and medial fusiform gyrus was observed to negative vs. neutral objects, but not to fearful vs. neutral faces. The current data suggest that the amygdala is responsive to animate and emotional stimuli. Additionally, these data suggest that the interaction between the various functions of the amygdala may need to be considered simultaneously to fully understand how they interact. Moreover, they suggest category-specific modulation of medial fusiform cortex as a function of emotion.

Effective connectivity during animacy perception--dynamic causal modelling of Human Connectome Project data

Biological agents are the most complex systems humans have to model and predict. In predictive coding, high-level cortical areas inform sensory cortex about incoming sensory signals, a comparison between the predicted and actual sensory feedback is made, and information about unpredicted sensory information is passed forward to higher-level areas. Predictions about animate motion - relative to inanimate motion - should result in prediction error and increase signal passing from lower level sensory area MT+/V5, which is responsive to all motion, to higher-order posterior superior temporal sulcus (pSTS), which is selectively activated by animate motion. We tested this hypothesis by investigating effective connectivity in a large-scale fMRI dataset from the Human Connectome Project. 132 participants viewed animations of triangles that were designed to move in a way that appeared animate (moving intentionally), or inanimate (moving in a mechanical way). We found that forward connectivity from V5 to the pSTS increased, and inhibitory self-connection in the pSTS decreased, when viewing intentional motion versus inanimate motion. These prediction errors associated with animate motion may be the cause for increased attention to animate stimuli found in previous studies.

Simultaneous EEG-fMRI reveals temporal evolution of coupling between supramodal cortical attention networks and the brainstem

Cortical and subcortical networks have been identified that are commonly associated with attention and task engagement, along with theories regarding their functional interaction. However, a link between these systems has not yet been demonstrated in healthy humans, primarily because of data acquisition and analysis limitations. We recorded simultaneous EEG-fMRI while subjects performed auditory and visual oddball tasks and used these data to investigate the BOLD correlates of single-trial EEG variability at latencies spanning the trial. We focused on variability along task-relevant dimensions in the EEG for identical stimuli and then combined auditory and visual data at the subject level to spatially and temporally localize brain regions involved in endogenous attentional modulations. Specifically, we found that anterior cingulate cortex (ACC) correlates strongly with both early and late EEG components, whereas brainstem, right middle frontal gyrus (rMFG), and right orbitofrontal cortex (rOFC) correlate significantly only with late components. By orthogonalizing with respect to event-related activity, we found that variability in insula and temporoparietal junction is reflected in reaction time variability, rOFC and brainstem correlate with residual EEG variability, and ACC and rMFG are significantly correlated with both. To investigate interactions between these correlates of temporally specific EEG variability, we performed dynamic causal modeling (DCM) on the fMRI data. We found strong evidence for reciprocal effective connections between the brainstem and cortical regions. Our results support the adaptive gain theory of locus ceruleus-norepinephrine (LC-NE) function and the proposed functional relationship between the LC-NE system, right-hemisphere ventral attention network, and P300 EEG response.