Cooperation of different neuronal systems during hand sign recognition

Study of aged central auditory function using the auditory middle latency response

OBJECTIVE

Investigate the auditory function of the elderly using the middle latency potentials.

METHODOLOGY

Group 1 (G1): 20 healthy individuals of both genders, older than 60 years, without hearing loss. Group 2 (G2): 20 healthy individuals of both sexes, older than 60 years, with hearing loss in frequencies from 4 to 8 kHz. Potential recording was performed with unilateral and bilateral stimulation and the Binaural Interaction Component was calculated.

RESULTS

Na latency in C3A1 was greater in the stimulation of the right ear in G2 and the amplitude of Na-Pa was greater in the stimulation of the right ear and recording in C3A1 in G1. The latency of the Pa component was higher in the stimulation of the right ear recorded in C4A2. The Pb component in G2 by bilateral stimulation and recorded in C4A2 had higher latency. The first and second negative and positive peaks presented greater amplitude in G1. In C3A1, the 1^st negative peak was more negative in G1 and the 2^nd positive peak showed greater amplitude in C4A2 in both groups.

CONCLUSION

The transmission of auditory information to the primary auditory cortex is impaired with aging, especially in unilateral stimulation, reinforced by losses in elderly people with peripheral hearing loss, such as in the binaural interaction at the cortical and subcortical levels. Thus, the AMLR has shown to be a sensitive examination to investigate neuroauditory disorders in the elderly, especially related to high-frequency hearing loss and primary auditory cortex dysfunctions caused by the aging process.

Electrophysiological evidence for internalized representations of canonical finger-number gestures and their facilitating effects on adults' math verification performance

Fingers facilitate number learning and arithmetic processing in early childhood. The current study investigated whether images of early-learned, culturally-typical (canonical), finger montring patterns presenting smaller (2,3,4) or larger (7,8,9) quantities still facilitate adults' performance and neural processing in a math verification task. Twenty-eight adults verified solutions to simple addition problems that were shown in the form of canonical or non-canonical finger-number montring patterns while measuring Event Related Potentials (ERPs). Results showed more accurate and faster sum verification when sum solutions were shown by canonical (versus non-canonical) finger patterns. Canonical finger montring patterns 2-4 led to faster responses independent of whether they presented correct or incorrect sum solutions and elicited an enhanced early right-parietal P2p response, whereas canonical configurations 7-9 only facilitated performance in correct sum solution trials without evoking P2p effects. The later central-parietal P3 was enhanced to all canonical finger patterns irrespective of numerical range. These combined results provide behavioral and brain evidence for canonical cardinal finger patterns still having facilitating effects on adults' number processing. They further suggest that finger montring configurations of numbers 2-4 have stronger internalized associations with other magnitude representations, possibly established through their mediating role in the developmental phase in which children acquire the numerical meaning of the first four number symbols.

The effect of interbrain synchronization in gesture observation: A fNIRS study

INTRODUCTION

Gestures characterize individuals' nonverbal communicative exchanges, taking on different functions. Several types of research in the neuroscientific field have been interested in the investigation of the neural correlates underlying the observation and implementation of different gestures categories. In particular, different studies have focused on the neural correlates underlying gestures observation, emphasizing the presence of mirroring mechanisms in specific brain areas, which appear to be involved in gesture observation and planning mechanisms.

MATERIALS AND METHODS

Specifically, the present study aimed to investigate the neural mechanisms, through the use of functional Near-Infrared Spectroscopy (fNIRS), underlying the observation of affective, social, and informative gestures with positive and negative valence in individuals' dyads composed by encoder and decoder. The variations of oxygenated (O2Hb) and deoxygenated (HHb) hemoglobin concentrations of both individuals were collected simultaneously through the use of hyperscanning paradigm, allowing the recording of brain responsiveness and interbrain connectivity.

RESULTS

The results showed a different brain activation and an increase of interbrain connectivity according to the type of gestures observed, with a significant increase of O2Hb brain responsiveness and interbrain connectivity and a decrease of HHb brain responsiveness for affective gestures in the dorsolateral prefrontal cortex (DLPFC) and for social gestures in the superior frontal gyrus (SFG). Furthermore, concerning the valence of the observed gestures, an increase of O2Hb brain activity and interbrain connectivity was observed in the left DLPFC for positive affective gestures compared to negative ones.

CONCLUSION

In conclusion, the present study showed different brain responses underlying the observation of different types of positive and negative gestures. Moreover, interbrain connectivity calculation allowed us to underline the presence of mirroring mechanisms involved in gesture-specific frontal regions during gestures observation and action planning.

Affective, Social, and Informative Gestures Reproduction in Human Interaction: Hyperscanning and Brain Connectivity

Gestural communication characterizes daily individuals' interactions in order to share information and to modify others' behavior. Social neuroscience has investigated the neural bases which support recognizing of different gestures. The present research, through the use of the hyperscanning approach, that allows the simultaneously recording of the activity of two or more individuals involved in a joint action, aims to investigate the neural bases of gestural communication. Moreover, by using hyperscanning paradigm we explore the inter-brain connectivity between two inter-agents, the one who performed the gesture (encoder) and the one who received it (decoder), with functional Near-infrared Spectroscopy (fNIRS) during the reproduction of affective, social and informative gestures with positive and negative valence. Result showed an increase in oxygenated hemoglobin concentration (O2Hb) and inter-brain connectivity in the dorsolateral prefrontal cortex (DLPFC) for affective gestures, in the superior frontal gyrus (SFG) for social gestures and the frontal eye fields (FEF) for informative gestures, for both encoder and decoder. Furthermore, it emerged that positive gestures activate more the left DLPFC, with an increase in inter-brain connectivity in DLPFC and SFG. The present study revealed the relevant function of the type and valence of gestures in affecting intra- and inter-brain connectivity.

Adult Attachment Affects Neural Response to Preference-Inferring in Ambiguous Scenarios: Evidence From an fMRI Study

Humans are highly social animals, and the ability to cater to the preferences of other individuals is encouraged by society. Preference-inferring is an important aspect of the theory of mind (TOM). Many previous studies have shown that attachment style is closely related to TOM ability. However, little is known about the effects of adult attachment style on preferences inferring under different levels of certainty. Here, we investigated how adult attachment style affects neural activity underlying preferences inferred under different levels of certainty by using functional magnetic resonance imaging (fMRI). The fMRI results demonstrated that adult attachment influenced the activation of anterior insula (AI) and inferior parietal lobule (IPL) in response to ambiguous preference-inferring. More specifically, in the ambiguous preference condition, the avoidant attached groups exhibited a significantly enhanced activation than secure and anxious attached groups in left IPL; the anxious attached groups exhibited a significantly reduced activation secure attached group in left IPL. In addition, the anxious attached groups exhibited a significantly reduced activation than secure and avoidant attached groups in left AI. These results were also further confirmed by the subsequent PPI analysis. The results from current study suggest that, under ambiguous situations, the avoidant attached individuals show lower sensitivity to the preference of other individuals and need to invest more cognitive resources for preference-reasoning; while compared with avoidant attached group, the anxious attached individuals express high tolerance for uncertainty and a higher ToM proficiency. Results from the current study imply that differences in preference-inferring under ambiguous conditions associated with different levels of individual attachment may explain the differences in interpersonal interaction.

Perceived Conventionality in Co-speech Gestures Involves the Fronto-Temporal Language Network

Face-to-face communication is multimodal; it encompasses spoken words, facial expressions, gaze, and co-speech gestures. In contrast to linguistic symbols (e.g., spoken words or signs in sign language) relying on mostly explicit conventions, gestures vary in their degree of conventionality. Bodily signs may have a general accepted or conventionalized meaning (e.g., a head shake) or less so (e.g., self-grooming). We hypothesized that subjective perception of conventionality in co-speech gestures relies on the classical language network, i.e., the left hemispheric inferior frontal gyrus (IFG, Broca's area) and the posterior superior temporal gyrus (pSTG, Wernicke's area) and studied 36 subjects watching video-recorded story retellings during a behavioral and an functional magnetic resonance imaging (fMRI) experiment. It is well documented that neural correlates of such naturalistic videos emerge as intersubject covariance (ISC) in fMRI even without involving a stimulus (model-free analysis). The subjects attended either to perceived conventionality or to a control condition (any hand movements or gesture-speech relations). Such tasks modulate ISC in contributing neural structures and thus we studied ISC changes to task demands in language networks. Indeed, the conventionality task significantly increased covariance of the button press time series and neuronal synchronization in the left IFG over the comparison with other tasks. In the left IFG, synchronous activity was observed during the conventionality task only. In contrast, the left pSTG exhibited correlated activation patterns during all conditions with an increase in the conventionality task at the trend level only. Conceivably, the left IFG can be considered a core region for the processing of perceived conventionality in co-speech gestures similar to spoken language. In general, the interpretation of conventionalized signs may rely on neural mechanisms that engage during language comprehension.

When to Take a Gesture Seriously: On How We Use and Prioritize Communicative Cues

When people talk, their speech is often accompanied by gestures. Although it is known that co-speech gestures can influence face-to-face communication, it is currently unclear to what extent they are actively used and under which premises they are prioritized to facilitate communication. We investigated these open questions in two experiments that varied how pointing gestures disambiguate the utterances of an interlocutor. Participants, whose event-related brain responses were measured, watched a video, where an actress was interviewed about, for instance, classical literature (e.g., Goethe and Shakespeare). While responding, the actress pointed systematically to the left side to refer to, for example, Goethe, or to the right to refer to Shakespeare. Her final statement was ambiguous and combined with a pointing gesture. The P600 pattern found in Experiment 1 revealed that, when pointing was unreliable, gestures were only monitored for their cue validity and not used for reference tracking related to the ambiguity. However, when pointing was a valid cue (Experiment 2), it was used for reference tracking, as indicated by a reduced N400 for pointing. In summary, these findings suggest that a general prioritization mechanism is in use that constantly monitors and evaluates the use of communicative cues against communicative priors on the basis of accumulated error information.

From perceptual to lexico-semantic analysis--cortical plasticity enabling new levels of processing

Certain kinds of stimuli can be processed on multiple levels. While the neural correlates of different levels of processing (LOPs) have been investigated to some extent, most of the studies involve skills and/or knowledge already present when performing the task. In this study we specifically sought to identify neural correlates of an evolving skill that allows the transition from perceptual to a lexico‐semantic stimulus analysis. Eighteen participants were trained to decode 12 letters of Morse code that were presented acoustically inside and outside of the scanner environment. Morse code was presented in trains of three letters while brain activity was assessed with fMRI. Participants either attended to the stimulus length (perceptual analysis), or evaluated its meaning distinguishing words from nonwords (lexico‐semantic analysis). Perceptual and lexico‐semantic analyses shared a mutual network comprising the left premotor cortex, the supplementary motor area (SMA) and the inferior parietal lobule (IPL). Perceptual analysis was associated with a strong brain activation in the SMA and the superior temporal gyrus bilaterally (STG), which remained unaltered from pre and post training. In the lexico‐semantic analysis post learning, study participants showed additional activation in the left inferior frontal cortex (IFC) and in the left occipitotemporal cortex (OTC), regions known to be critically involved in lexical processing. Our data provide evidence for cortical plasticity evolving with a learning process enabling the transition from perceptual to lexico‐semantic stimulus analysis. Importantly, the activation pattern remains task‐related LOP and is thus the result of a decision process as to which LOP to engage in. Hum Brain Mapp 36:4512–4528, 2015. © 2015 The Authors. Human Brain Mapping Published byWiley Periodicals, Inc.

P300 in individuals with sensorineural hearing loss

Introduction

Behavioral and electrophysiological auditory evaluations contribute to the understanding of the auditory system and of the process of intervention.

Objective

To study P300 in subjects with severe or profound sensorineural hearing loss.

Methods

This was a descriptive cross-sectional prospective study. It included 29 individuals of both genders with severe or profound sensorineural hearing loss without other type of disorders, aged 11 to 42 years; all were assessed by behavioral audiological evaluation and auditory evoked potentials.

Results

A recording of the P3 wave was obtained in 17 individuals, with a mean latency of 326.97 ms and mean amplitude of 3.76 V. There were significant differences in latency in relation to age and in amplitude according to degree of hearing loss. There was a statistically significant association of the P300 results with the degrees of hearing loss (p = 0.04), with the predominant auditory communication channels (p < 0.0001), and with time of hearing loss.

Conclusions

P300 can be recorded in individuals with severe and profound congenital sensorineural hearing loss; it may contribute to the understanding of cortical development and is a good predictor of the early intervention outcome.

Functional connectivity in task-negative network of the Deaf: effects of sign language experience

Prior studies investigating cortical processing in Deaf signers suggest that life-long experience with sign language and/or auditory deprivation may alter the brain’s anatomical structure and the function of brain regions typically recruited for auditory processing (Emmorey et al., 2010; Pénicaud et al., 2013 inter alia). We report the first investigation of the task-negative network in Deaf signers and its functional connectivity—the temporal correlations among spatially remote neurophysiological events. We show that Deaf signers manifest increased functional connectivity between posterior cingulate/precuneus and left medial temporal gyrus (MTG), but also inferior parietal lobe and medial temporal gyrus in the right hemisphere- areas that have been found to show functional recruitment specifically during sign language processing. These findings suggest that the organization of the brain at the level of inter-network connectivity is likely affected by experience with processing visual language, although sensory deprivation could be another source of the difference. We hypothesize that connectivity alterations in the task negative network reflect predictive/automatized processing of the visual signal.

Spatiotemporal dynamics of early cortical gesture processing

Gesture processing has been consistently shown to be associated with activation of the inferior parietal lobe (IPL); however, little is known about the integration of IPL activation into the temporal dynamics of early sensory areas. Using a temporally graded repetition suppression paradigm, we examined the activation and time course of brain areas involved in hand gesture processing. We recorded event-related potentials in response to stimulus pairs of static hand images forming gestures of the popular rock-paper-scissors game and estimated their neuronal generators. We identified two main components associated with adaptive patterns related to stimulus repetition. The N190 component elicited at temporo-parietal sites adapted to repetitions of the same gesture and was associated with right-hemispheric extrastriate body area activation. A later component at parieto-occipital sites demonstrated temporally graded adaptation effects for all gestures with a left-hemispheric dominance. Source localization revealed concurrent activations of the right extrastriate body area, fusiform gyri bilaterally, and the left IPL at about 250 ms. The adaptation pattern derived from the graded repetition suppression paradigm demonstrates the functional sensitivity of these sources to gesture processing. Given the literature on IPL contribution to imitation, action recognition, and action execution, IPL activation at about 250 ms may represent the access into specific cognitive routes for gesture processing and may thus be involved in integrating sensory information from cortical body areas into subsequent visuo-motor transformation processes.

Different hemispheric roles in recognition of happy expressions

The emotional expression of the face provides an important social signal that allows humans to make inferences about other people's state of mind. However, the underlying brain mechanisms are complex and still not completely understood. Using magnetoencephalography (MEG), we analyzed the spatiotemporal structure of regional electrical brain activity in human adults during a categorization task (faces or hands) and an emotion discrimination task (happy faces or neutral faces). Brain regions that are specifically important for different aspects of processing emotional facial expressions showed interesting hemispheric dominance patterns. The dorsal brain regions showed a right predominance when participants paid attention to facial expressions: The right parietofrontal regions, including the somatosensory, motor/premotor, and inferior frontal cortices showed significantly increased activation in the emotion discrimination task, compared to in the categorization task, in latencies of 350 to 550 ms, while no activation was found in their left hemispheric counterparts. Furthermore, a left predominance of the ventral brain regions was shown for happy faces, compared to neutral faces, in latencies of 350 to 550 ms within the emotion discrimination task. Thus, the present data suggest that the right and left hemispheres play different roles in the recognition of facial expressions depending on cognitive context.

Supramodal neural processing of abstract information conveyed by speech and gesture

Abstractness and modality of interpersonal communication have a considerable impact on comprehension. They are relevant for determining thoughts and constituting internal models of the environment. Whereas concrete object-related information can be represented in mind irrespective of language, abstract concepts require a representation in speech. Consequently, modality-independent processing of abstract information can be expected. Here we investigated the neural correlates of abstractness (abstract vs. concrete) and modality (speech vs. gestures), to identify an abstractness-specific supramodal neural network. During fMRI data acquisition 20 participants were presented with videos of an actor either speaking sentences with an abstract-social [AS] or concrete-object-related content [CS], or performing meaningful abstract-social emblematic [AG] or concrete-object-related tool-use gestures [CG]. Gestures were accompanied by a foreign language to increase the comparability between conditions and to frame the communication context of the gesture videos. Participants performed a content judgment task referring to the person vs. object-relatedness of the utterances. The behavioral data suggest a comparable comprehension of contents communicated by speech or gesture. Furthermore, we found common neural processing for abstract information independent of modality (AS > CS ∩ AG > CG) in a left hemispheric network including the left inferior frontal gyrus (IFG), temporal pole, and medial frontal cortex. Modality specific activations were found in bilateral occipital, parietal, and temporal as well as right inferior frontal brain regions for gesture (G > S) and in left anterior temporal regions and the left angular gyrus for the processing of speech semantics (S > G). These data support the idea that abstract concepts are represented in a supramodal manner. Consequently, gestures referring to abstract concepts are processed in a predominantly left hemispheric language related neural network.

A supramodal neural network for speech and gesture semantics: an fMRI study

In a natural setting, speech is often accompanied by gestures. As language, speech-accompanying iconic gestures to some extent convey semantic information. However, if comprehension of the information contained in both the auditory and visual modality depends on same or different brain-networks is quite unknown. In this fMRI study, we aimed at identifying the cortical areas engaged in supramodal processing of semantic information. BOLD changes were recorded in 18 healthy right-handed male subjects watching video clips showing an actor who either performed speech (S, acoustic) or gestures (G, visual) in more (+) or less (−) meaningful varieties. In the experimental conditions familiar speech or isolated iconic gestures were presented; during the visual control condition the volunteers watched meaningless gestures (G−), while during the acoustic control condition a foreign language was presented (S−). The conjunction of the visual and acoustic semantic processing revealed activations extending from the left inferior frontal gyrus to the precentral gyrus, and included bilateral posterior temporal regions. We conclude that proclaiming this frontotemporal network the brain's core language system is to take too narrow a view. Our results rather indicate that these regions constitute a supramodal semantic processing network.

Gesture's Neural Language

When people talk to each other, they often make arm and hand movements that accompany what they say. These manual movements, called “co-speech gestures,” can convey meaning by way of their interaction with the oral message. Another class of manual gestures, called “emblematic gestures” or “emblems,” also conveys meaning, but in contrast to co-speech gestures, they can do so directly and independent of speech. There is currently significant interest in the behavioral and biological relationships between action and language. Since co-speech gestures are actions that rely on spoken language, and emblems convey meaning to the effect that they can sometimes substitute for speech, these actions may be important, and potentially informative, examples of language–motor interactions. Researchers have recently been examining how the brain processes these actions. The current results of this work do not yet give a clear understanding of gesture processing at the neural level. For the most part, however, it seems that two complimentary sets of brain areas respond when people see gestures, reflecting their role in disambiguating meaning. These include areas thought to be important for understanding actions and areas ordinarily related to processing language. The shared and distinct responses across these two sets of areas during communication are just beginning to emerge. In this review, we talk about the ways that the brain responds when people see gestures, how these responses relate to brain activity when people process language, and how these might relate in normal, everyday communication.

Sensorimotor Alpha Activity is Modulated in Response to the Observation of Pain in Others

The perception-action account of empathy states that observation of another person's state automatically activates a similar state in the observer. It is still unclear in what way ongoing sensorimotor alpha oscillations are involved in this process. Although they have been repeatedly implicated in (biological) action observation and understanding communicative gestures, less is known about their role in vicarious pain observation. Their role is understood as providing a graded inhibition through functional inhibition, thereby streamlining information flow through the cortex. Although alpha oscillations have been shown to have at least visual and sensorimotor origins, only the latter are expected to be involved in the empathetic response. Here, we used magnetoencephalography, allowing us to spatially distinguish and localize oscillatory components using beamformer source reconstruction. Subjects observed realistic pictures of limbs in painful and no-pain (control) conditions. As predicted, time-frequency analysis indeed showed increased alpha suppression in the pain condition compared to the no-pain condition. Although both pain and no-pain conditions suppressed alpha- and beta-band activity at both posterior and central sensors, the pain condition suppressed alpha more only at central sensors. Source reconstruction localized these differences along the central sulcus. Our results could not be accounted for by differences in the evoked fields, suggesting a unique role of oscillatory activity in empathetic responses. We argue that alpha oscillations provide a unique measure of the underlying functional architecture of the brain, suggesting an automatic disinhibition of the sensorimotor cortices in response to the observation of pain in others.

Intention Processing in Communication: A Common Brain Network for Language and Gestures

Human communicative competence is based on the ability to process a specific class of mental states, namely, communicative intention. The present fMRI study aims to analyze whether intention processing in communication is affected by the expressive means through which a communicative intention is conveyed, that is, the linguistic or extralinguistic gestural means. Combined factorial and conjunction analyses were used to test two sets of predictions: first, that a common brain network is recruited for the comprehension of communicative intentions independently of the modality through which they are conveyed; second, that additional brain areas are specifically recruited depending on the communicative modality used, reflecting distinct sensorimotor gateways. Our results clearly showed that a common neural network is engaged in communicative intention processing independently of the modality used. This network includes the precuneus, the left and right posterior STS and TPJ, and the medial pFC. Additional brain areas outside those involved in intention processing are specifically engaged by the particular communicative modality, that is, a peri-sylvian language network for the linguistic modality and a sensorimotor network for the extralinguistic modality. Thus, common representation of communicative intention may be accessed by modality-specific gateways, which are distinct for linguistic versus extralinguistic expressive means. Taken together, our results indicate that the information acquired by different communicative modalities is equivalent from a mental processing standpoint, in particular, at the point at which the actor's communicative intention has to be reconstructed.

Time course and specificity of sensory-motor alpha modulation during the observation of hand motor acts and gestures: a high density EEG study

The main aim of the present study was to explore, by means of high-density EEG, the intensity and the temporal pattern of event-related sensory-motor alpha desynchronization (ERD) during the observation of different types of hand motor acts and gestures. In particular, we aimed to investigate whether the sensory-motor ERD would show a specific modulation during the observation of hand behaviors differing for goal-relatedness (hand grasping of an object and meaningless hand movements) and social relevance (communicative hand gestures and grasping within a social context). Time course analysis of alpha suppression showed that all types of hand behaviors were effective in triggering sensory-motor alpha ERD, but to a different degree depending on the category of observed hand motor acts and gestures. Meaningless gestures and hand grasping were the most effective stimuli, resulting in the strongest ERD. The observation of social hand behaviors such as social grasping and communicative gestures, triggered a more dynamic time course of ERD compared to that driven by the observation of simple grasping and meaningless gestures. These findings indicate that the observation of hand motor acts and gestures evoke the activation of a motor resonance mechanism that differs on the basis of the goal-relatedness and the social relevance of the observed hand behavior.

Commonalities in the neural mechanisms underlying automatic attentional shifts by gaze, gestures, and symbols

Eye gaze, hand-pointing gestures, and arrows automatically trigger attentional shifts. Although it has been suggested that common neural mechanisms underlie these three types of attentional shifts, this issue remains unsettled. We measured brain activity using fMRI while participants observed directional and non-directional stimuli, including eyes, hands, and arrows, to investigate this issue. Conjunction analyses revealed that the posterior superior temporal sulcus (STS), the inferior parietal lobule, the inferior frontal gyrus, and the occipital cortices in the right hemisphere were more active in common in response to directional versus non-directional stimuli. These results suggest commonalities in the neurocognitive mechanisms underlying the automatic attentional shifts triggered by gaze, gestures, and symbols.

N400-like negativities in action perception reflect the activation of two components of an action representation

The understanding of actions of tool use depends on the motor act that is performed and on the function of the objects involved in the action. We used event-related potentials (ERPs) to investigate the processes that derive both kinds of information in a task in which inserting actions had to be judged. The actions were presented as two consecutive frames, one showing an effector/instrument and the other showing a potential target object of the action. Two mismatches were possible. An orientation mismatch occurred when the spatial object properties were not consistent with a motor act of insertion being performed (i.e., different orientations of insert and slot). A functional mismatch happened when the instrument (e.g., screwdriver) would usually not be applied to the target object (e.g., keyhole). The order in which instrument and target object were presented was also varied. The two kinds of mismatch gave rise to similar but not identical negativities in the latency range of the N400 followed by a positive modulation. The results indicate that the motor act and the function of the objects are derived by two at least partially different subprocesses and become integrated into a common representation of the observed action.

Cross-cultural music phrase processing: an fMRI study

The current study used functional magnetic resonance imaging (fMRI) to investigate the neural basis of musical phrase boundary processing during the perception of music from native and non-native cultures. German musicians performed a cultural categorization task while listening to phrased Western (native) and Chinese (non-native) musical excerpts as well as modified versions of these, where the impression of phrasing has been reduced by removing the phrase boundary marking pause (henceforth called "unphrased"). Bilateral planum temporale was found to be associated with an increased difficulty of identifying phrase boundaries in unphrased Western melodies. A network involving frontal and parietal regions showed increased activation for the phrased condition with the orbital part of left inferior frontal gyrus presumably reflecting working memory aspects of the temporal integration between phrases, and the middle frontal gyrus and intraparietal sulcus probably reflecting attention processes. Areas more active in the culturally familiar, native (Western) condition included, in addition to the left planum temporale and right ventro-medial prefrontal cortex, mainly the bilateral motor regions. These latter results are interpreted in light of sensorimotor integration. Regions with increased signal for the unfamiliar, non-native music style (Chinese) included a right lateralized network of angular gyrus and the middle frontal gyrus, possibly reflecting higher demands on attention systems, and the right posterior insula suggesting higher loads on basic auditory processing.

Do you see what I mean? Corticospinal excitability during observation of culture-specific gestures

People all over the world use their hands to communicate expressively. Autonomous gestures, also known as emblems, are highly social in nature, and convey conventionalized meaning without accompanying speech. To study the neural bases of cross-cultural social communication, we used single pulse transcranial magnetic stimulation (TMS) to measure corticospinal excitability (CSE) during observation of culture-specific emblems. Foreign Nicaraguan and familiar American emblems as well as meaningless control gestures were performed by both a Euro-American and a Nicaraguan actor. Euro-American participants demonstrated higher CSE during observation of the American compared to the Nicaraguan actor. This motor resonance phenomenon may reflect ethnic and cultural ingroup familiarity effects. However, participants also demonstrated a nearly significant (p = 0.053) actor by emblem interaction whereby both Nicaraguan and American emblems performed by the American actor elicited similar CSE, whereas Nicaraguan emblems performed by the Nicaraguan actor yielded higher CSE than American emblems. The latter result cannot be interpreted simply as an effect of ethnic ingroup familiarity. Thus, a likely explanation of these findings is that motor resonance is modulated by interacting biological and cultural factors.

The superior temporal sulcus performs a common function for social and speech perception: implications for the emergence of autism

Within the cognitive neuroscience literature, discussion of the functional role of the superior temporal sulcus (STS) has traditionally been divided into two domains; one focuses on its activity during language processing while the other emphasizes its role in biological motion and social attention, such as eye gaze processing. I will argue that a common process underlying both of these functional domains is performed by the STS, namely analyzing changing sequences of input, either in the auditory or visual domain, and interpreting the communicative significance of those inputs. From a developmental perspective, the fact that these two domains share an anatomical substrate suggests the acquisition of social and speech perception may be linked. In addition, I will argue that because of the STS' role in interpreting social and speech input, impairments in STS function may underlie many of the social and language abnormalities seen in autism.

Neural evidence for the interplay between language, gesture, and action: a review

Co-speech gestures embody a form of manual action that is tightly coupled to the language system. As such, the co-occurrence of speech and co-speech gestures is an excellent example of the interplay between language and action. There are, however, other ways in which language and action can be thought of as closely related. In this paper we will give an overview of studies in cognitive neuroscience that examine the neural underpinnings of links between language and action. Topics include neurocognitive studies of motor representations of speech sounds, action-related language, sign language and co-speech gestures. It will be concluded that there is strong evidence on the interaction between speech and gestures in the brain. This interaction however shares general properties with other domains in which there is interplay between language and action.

An intentional stance modulates the integration of gesture and speech during comprehension

The present study investigates whether knowledge about the intentional relationship between gesture and speech influences controlled processes when integrating the two modalities at comprehension. Thirty-five adults watched short videos of gesture and speech that conveyed semantically congruous and incongruous information. In half of the videos, participants were told that the two modalities were intentionally coupled (i.e., produced by the same communicator), and in the other half, they were told that the two modalities were not intentionally coupled (i.e., produced by different communicators). When participants knew that the same communicator produced the speech and gesture, there was a larger bi-lateral frontal and central N400 effect to words that were semantically incongruous versus congruous with gesture. However, when participants knew that different communicators produced the speech and gesture--that is, when gesture and speech were not intentionally meant to go together--the N400 effect was present only in right-hemisphere frontal regions. The results demonstrate that pragmatic knowledge about the intentional relationship between gesture and speech modulates controlled neural processes during the integration of the two modalities.

Children's and adults' neural bases of verbal and nonverbal 'theory of mind'

Theory of mind (ToM) - our ability to predict behaviors of others in terms of their underlying intentions - has been examined through verbal and nonverbal false-belief (FB) tasks. Previous brain imaging studies of ToM in adults have implicated medial prefrontal cortex (mPFC) and temporo-parietal junction (TPJ) for adults' ToM ability. To examine age and modality related differences and similarities in neural correlates of ToM, we tested 16 adults (18-40 years old) and 12 children (8-12 years old) with verbal (story) and nonverbal (cartoon) FB tasks, using functional magnetic resonance imaging (fMRI). Both age groups showed significant activity in the TPJ bilaterally and right inferior parietal lobule (IPL) in a modality-independent manner, indicating that these areas are important for ToM during both adulthood and childhood, regardless of modality. We also found significant age-related differences in the ToM condition-specific activity for the story and cartoon tasks in the left inferior frontal gyrus (IFG) and left TPJ. These results suggest that depending on the modality adults may utilize different brain regions from children in understanding ToM.

When Language Meets Action: The Neural Integration of Gesture and Speech

Although generally studied in isolation, language and action often co-occur in everyday life. Here we investigated one particular form of simultaneous language and action, namely speech and gestures that speakers use in everyday communication. In a functional magnetic resonance imaging study, we identified the neural networks involved in the integration of semantic information from speech and gestures. Verbal and/or gestural content could be integrated easily or less easily with the content of the preceding part of speech. Premotor areas involved in action observation (Brodmann area [BA] 6) were found to be specifically modulated by action information "mismatching" to a language context. Importantly, an increase in integration load of both verbal and gestural information into prior speech context activated Broca's area and adjacent cortex (BA 45/47). A classical language area, Broca's area, is not only recruited for language-internal processing but also when action observation is integrated with speech. These findings provide direct evidence that action and language processing share a high-level neural integration system.

Localizing the distributed language network responsible for the N400 measured by MEG during auditory sentence processing

We studied auditory sentence comprehension using magnetoencephalography while subjects listened to sentences whose correctness they had to judge subsequently. The localization and the time course of brain electrical activity during processing of correct and semantically incorrect sentences were estimated by computing a brain surface current density within a cortical layer for both conditions. Finally, a region of interest (ROI) analysis was conducted to determine the time course of specific locations. A magnetic N400 was present in six spatially different ROIs. Semantic anomalies caused an exclusive involvement of the ventral portion of the left inferior frontal gyrus (BA 47) and left pars triangularis (BA 45). The anterior parts of the superior (BA 22) and inferior (BA 20/21) temporal gyri bilaterally were activated by both conditions. The activation for the correct condition, however, peaked earlier in both left temporal regions (approximately 32 ms). In general, activation due to semantic violations was more pronounced, started later, and lasted longer as compared to correct sentences. The findings reveal a clear left-hemispheric dominance during language processing indicated firstly by the mere number of activated regions (four in the left vs. two in the right hemisphere) and secondly by the observed specificity of the left inferior frontal ROIs to semantic violations. The temporal advantage observed for the correct condition in the left temporal regions is supporting the notion that the established context eases the processing of the final word. Semantically incorrect words that do not fit into the context result in longer integration times.

Brain Networks Involved in Viewing Angry Hands or Faces

Most neuropsychological research on the perception of emotion concerns the perception of faces. Yet in everyday life, hand actions are also modulated by our affective state, revealing it, in turn, to the observer. We used functional magnetic resonance imaging (fMRI) to identify brain regions engaged during the observation of hand actions performed either in a neutral or an angry way. We also asked whether these are the same regions as those involved in perceiving expressive faces. During the passive observation of emotionally neutral hand movements, the fMRI signal increased significantly in dorsal and ventral premotor cortices, with the exact location of the 'peaks' distinct from those induced by face observation. Various areas in the extrastriate visual cortex were also engaged, overlapping with the face-related activity. When the observed hand action was performed with emotion, additional regions were recruited including the right dorsal premotor, the right medial prefrontal cortex, the left anterior insula and a region in the rostral part of the supramarginal gyrus bilaterally. These regions, except for the supramarginal gyrus, were also activated during the perception of angry faces. These results complement the wealth of studies on the perception of affect from faces and provide further insights into the processes involved in the perception of others underlying, perhaps, social constructs such as empathy.