Spontaneous gesture and spatial language: Evidence from focal brain injury

The Neural correlates of limb apraxia: An anatomical likelihood estimation meta-analysis of lesion-symptom mapping studies in brain-damaged patients

Limb apraxia is a motor disorder frequently observed following a stroke. Apraxic deficits are classically assessed with four tasks: tool use, pantomime of tool use, imitation, and gesture understanding. These tasks are supported by several cognitive processes represented in a left-lateralized brain network including inferior frontal gyrus, inferior parietal lobe (IPL), and lateral occipito-temporal cortex (LOTC). For the past twenty years, voxel-wise lesion symptom mapping (VLSM) studies have been used to unravel the neural correlates associated with apraxia, but none of them has proposed a comprehensive view of the topic. In the present work, we proposed to fill this gap by performing a systematic Anatomic Likelihood Estimation meta-analysis of VLSM studies which included tasks traditionally used to assess apraxia. We found that the IPL was crucial for all the tasks. Moreover, lesions within the LOTC were more associated with imitation deficits than tool use or pantomime, confirming its important role in higher visual processing. Our results questioned traditional neurocognitive models on apraxia and may have important clinical implications.

Temporal Overlap Between Gestures and Speech in Poststroke Aphasia: Is There a Compensatory Effect?

PURPOSE

If language production is impaired, will gestures compensate? Evidence in favor of this prediction has often been argued to come from aphasia, but it remains contested. Here, we tested whether thought content not present in speech due to language impairment is manifested in gestures, in 20 people with dysfluent (Broca's) aphasia, 20 people with fluent (Wernicke's) aphasia, and 20 matched neurotypical controls.

METHOD

A new annotation scheme was created distinguishing types of gestures and whether they co-occurred with fluent or dysfluent/absent speech and were temporally aligned in content with coproduced speech.

RESULTS

Across both aphasia types, noncontent (beat) gestures, which by their nature cannot compensate for lost speech content, constituted the greatest proportion of all types of gestures produced. Content (i.e., descriptive, referential, and metaphorical) gestures were largely coproduced with fluent rather than dysfluent speech and tended to be aligned with the content conveyed in speech. They also did not differ in quantity depending on whether the dysfluencies were eventually resolved or not. Neither aphasia severity nor comprehension ability had an impact on the total amount of content gesture produced in people with aphasia, which was instead positively correlated with speech fluency.

CONCLUSIONS

Together, these results suggest that gestures are unlikely to have a role in compensating for linguistic deficits and to serve as a representational system conveying thought content independent of language. Surprisingly, aphasia rather is a model of how gesture and language are inherently integrated and aligned: Even when language is impaired, it remains the essential provider of content.

Aging, Gesture Production, and Disfluency in Speech: A Comparison of Younger and Older Adults

Age-related changes are observed in the speech and gestures of neurotypical individuals. Older adults are more disfluent in speech and use fewer representational gestures (e.g., holding two hands close to each other to mean small), compared to younger adults. Using gestures, especially representational gestures, is common in difficult tasks to aid the conceptualization process and to facilitate lexical access. This study investigates how aging can affect gesture production and the co-occurrence between gesture and speech disfluency. We elicited speech and gesture samples from younger and older adults (N = 60) by using a painting description task that provided concrete and abstract contexts. Results indicated that albeit the two age groups revealed comparable overall speech disfluency and gesture rates, they differed in terms of how their disfluencies and gestures were distributed across specific categories. Moreover, the proportion of speech disfluencies that occur with a gesture was significantly higher for younger than older adults. However, the two age groups were comparable in terms of the proportion of gestures that were accompanied by a speech disfluency. These findings suggest that younger adults' language production system might be better at benefiting from other modalities, that is, gesture, to resolve temporary problems in speech planning. However, from a gesture perspective, it might be difficult to differentiate between gestures' self-oriented and communicative functions and understand their role in speech facilitation. Focusing on specific cases where speech disfluency and gestures co-occur and considering individual differences might bring insight into multimodal communication.

Production and comprehension of co-speech gestures in Parkinson's disease

This study examined how impairments in sensorimotor abilities of individuals with Parkinson's Disease (PD) can be related to the use and understanding of co-speech hand gestures involving literal and figurative actions. We tested individuals with PD (n = 18, 12 males, M_age = 56.5, SD_age = 8.16, PD duration since onset: M = 5.36 years, SD = 3.51, Hoehn and Yahr Scale:M_H&Y = 2.09, SD_H&Y = 0.50) and age- and education-matched neurotypical controls (n = 18, 14 males, M_age = 56.61, SD_age = 8.88) with two experimental tasks. In the gesture production task, participants retold the narratives presented to them in a written format. In the gesture comprehension task, participants were asked to match a gesture with a novel verb in literal and figurative sentence contexts. Results showed that patients with PD gestured significantly less than the neurotypical controls. No group differences were found for the type of gesture use. Individuals with PD performed worse than controls on matching gestures with novel verbs, particularly for figurative meanings. Individuals' severity in the disease negatively correlated with their performance for these figurative novel verb-gesture matches. The performances in the two tasks did not correlate. These findings suggest that problems in sensorimotor abilities resulting from PD can influence overall gesture production and gesture comprehension, providing further evidence on the relations between PD and the impaired use of multimodal language.

Suggestions for Improving the Investigation of Gesture in Aphasia

Purpose When we speak, we gesture, and indeed, persons with aphasia gesture more frequently. The reason(s) for this is still being investigated, spurring an increase in the number of studies of gesture in persons with aphasia. As the number of studies increases, so too does the need for a shared set of best practices for gesture research in aphasia. After briefly reviewing the importance and use of gesture in persons with aphasia, this viewpoint puts forth methodological and design considerations when evaluating gesture in persons with aphasia. Method & Results We explore several different design and methodological considerations for gesture research specific to persons with aphasia, such as video angle specifications, data collection techniques, and analysis considerations. The goal of these suggestions is to develop transparent and reproducible methods for evaluating gesture in aphasia to build a solid foundation for continued work in this area. Conclusions We have proposed that it is critical to evaluate multimodal communication in a methodologically robust way to facilitate increased knowledge about the relationship of gesture to spoken language, cognition, and to other aspects of living with aphasia and recovery from aphasia. We conclude by postulating future directions for gesture research in aphasia.

Gesture Use and Processing: A Review on Individual Differences in Cognitive Resources

Speakers use spontaneous hand gestures as they speak and think. These gestures serve many functions for speakers who produce them as well as for listeners who observe them. To date, studies in the gesture literature mostly focused on group-comparisons or the external sources of variation to examine when people use, process, and benefit from using and observing gestures. However, there are also internal sources of variation in gesture use and processing. People differ in how frequently they use gestures, how salient their gestures are, for what purposes they produce gestures, and how much they benefit from using and seeing gestures during comprehension and learning depending on their cognitive dispositions. This review addresses how individual differences in different cognitive skills relate to how people employ gestures in production and comprehension across different ages (from infancy through adulthood to healthy aging) from a functionalist perspective. We conclude that speakers and listeners can use gestures as a compensation tool during communication and thinking that interacts with individuals' cognitive dispositions.

The Production of Gesture and Speech by People With Aphasia: Influence of Communicative Constraints

Purpose People with aphasia (PWA) use different kinds of gesture spontaneously when they communicate. Although there is evidence that the nature of the communicative task influences the linguistic performance of PWA, so far little is known about the influence of the communicative task on the production of gestures by PWA. We aimed to investigate the influence of varying communicative constraints on the production of gesture and spoken expression by PWA in comparison to persons without language impairment. Method Twenty-six PWA with varying aphasia severities and 26 control participants (CP) without language impairment participated in the study. Spoken expression and gesture production were investigated in 2 different tasks: (a) spontaneous conversation about topics of daily living and (b) a cartoon narration task, that is, retellings of short cartoon clips. The frequencies of words and gestures as well as of different gesture types produced by the participants were analyzed and tested for potential effects of group and task. Results Main results for task effects revealed that PWA and CP used more iconic gestures and pantomimes in the cartoon narration task than in spontaneous conversation. Metaphoric gestures, deictic gestures, number gestures, and emblems were more frequently used in spontaneous conversation than in cartoon narrations by both participant groups. Group effects show that, in both tasks, PWA's gesture-to-word ratios were higher than those for the CP. Furthermore, PWA produced more interactive gestures than the CP in both tasks, as well as more number gestures and pantomimes in spontaneous conversation. Conclusions The current results suggest that PWA use gestures to compensate for their verbal limitations under varying communicative constraints. The properties of the communicative task influence the use of different gesture types in people with and without aphasia. Thus, the influence of communicative constraints needs to be considered when assessing PWA's multimodal communicative abilities.

Differential roles of gestures on spatial language in neurotypical elderly adults and individuals with focal brain injury

Gestures might serve communicative functions by supplementing spoken expressions or restorative functions by facilitating speech production. Also, speakers with speech deficits use gestures to compensate for their speech impairments. In this study, we examined gesture use in speakers with and without speech impairments and how spoken spatial expressions changed when gestures were restrained. Six patients with speech problems and with left frontal and/or temporal lesions and 20 neurotypical controls described motion events in 3 different conditions (spontaneous gesture, only speech, and only gesture). In addition to the group analyses, we ran case analyses. Results showed that patients used more gestures compared to controls. Gestures served both communicative and restorative functions for patients whereas controls only used gestures for communicative purposes. Case analyses revealed that there were differential patterns among patients. Overall, gesture production is multifunctional and gestures serve different functions for different populations as well as within a population.

The relationship between co-speech gesture production and macrolinguistic discourse abilities in people with focal brain injury

Brain damage is associated with linguistic deficits and might alter co-speech gesture production. Gesture production after focal brain injury has been mainly investigated with respect to intrasentential rather than discourse-level linguistic processing. In this study, we examined 1) spontaneous gesture production patterns of people with left hemisphere damage (LHD) or right hemisphere damage (RHD) in a narrative setting, 2) the neural structures associated with deviations in spontaneous gesture production in these groups, and 3) the relationship between spontaneous gesture production and discourse level linguistic processes (narrative complexity and evaluation competence). Individuals with LHD or RHD (17 people in each group) and neurotypical controls (n = 13) narrated a story from a picture book. Results showed that increase in gesture production for LHD individuals was associated with less complex narratives and lesions of individuals who produced more gestures than neurotypical individuals overlapped in frontal-temporal structures and basal ganglia. Co-speech gesture production of RHD individuals positively correlated with their evaluation competence in narrative. Lesions of RHD individuals who produced more gestures overlapped in the superior temporal gyrus and the inferior parietal lobule. Overall, LHD individuals produced more gestures than neurotypical individuals. The groups did not differ in their use of different gesture forms except that LHD individuals produced more deictic gestures per utterance than RHD individuals and controls. Our findings are consistent with the hypothesis that co-speech gesture production interacts with macro-linguistic levels of discourse and this interaction is affected by the hemispheric lateralization of discourse abilities.

Guided Embodiment and Potential Applications of Tutor Systems in Language Instruction and Rehabilitation

Intelligent tutor systems (ITSs) in mobile devices take us through learning tasks and make learning ubiquitous, autonomous, and at low cost (Nye, 2015). In this paper, we describe guided embodiment as an ITS essential feature for second language learning (L2) and aphasia rehabilitation (ARe) that enhances efficiency in the learning process. In embodiment, cognitive processes, here specifically language (re)learning are grounded in actions and gestures (Pecher and Zwaan, 2005; Fischer and Zwaan, 2008; Dijkstra and Post, 2015). In order to guide users through embodiment, ITSs must track action and gesture, and give corrective feed-back to achieve the users' goals. Therefore, sensor systems are essential to guided embodiment. In the next sections, we describe sensor systems that can be implemented in ITS for guided embodiment.

Multimodal Communication in Aphasia: Perception and Production of Co-speech Gestures During Face-to-Face Conversation

The role of nonverbal communication in patients with post-stroke language impairment (aphasia) is not yet fully understood. This study investigated how aphasic patients perceive and produce co-speech gestures during face-to-face interaction, and whether distinct brain lesions would predict the frequency of spontaneous co-speech gesturing. For this purpose, we recorded samples of conversations in patients with aphasia and healthy participants. Gesture perception was assessed by means of a head-mounted eye-tracking system, and the produced co-speech gestures were coded according to a linguistic classification system. The main results are that meaning-laden gestures (e.g., iconic gestures representing object shapes) are more likely to attract visual attention than meaningless hand movements, and that patients with aphasia are more likely to fixate co-speech gestures overall than healthy participants. This implies that patients with aphasia may benefit from the multimodal information provided by co-speech gestures. On the level of co-speech gesture production, we found that patients with damage to the anterior part of the arcuate fasciculus showed a higher frequency of meaning-laden gestures. This area lies in close vicinity to the premotor cortex and is considered to be important for speech production. This may suggest that the use of meaning-laden gestures depends on the integrity of patients’ speech production abilities.

Use of co-verbal gestures during word-finding difficulty among Cantonese speakers with fluent aphasia and unimpaired controls

BACKGROUND

Co-verbal gestures refer to hand or arm movements made during speaking. Spoken language and gestures have been shown to be tightly integrated in human communication.

AIMS

The present study investigated whether co-verbal gesture use was associated with lexical retrieval in connected speech in unimpaired speakers and persons with aphasia (PWA).

METHODS & PROCEDURES

Narrative samples of 58 fluent PWA and 58 control speakers were extracted from Cantonese AphasiaBank. Based on the indicators of word-finding difficulty (WFD) in connected speech adapted from previous research, and a gesture annotation system with independent coding of gesture forms and functions, all WFD instances were identified. The presence and type of gestures accompanying each incident of WFD were then annotated. Finally, whether the use of gesture was accompanied by resolution of WFD, i.e., the corresponding target word could be retrieved, was examined.

OUTCOMES & RESULTS

Employment of co-verbal gesture did not seem to be related to the success of word retrieval. PWA's naming ability at single-word level and their overall language ability (as reflected by the aphasia quotient of the Cantonese version of the Western Aphasia Battery) were found to be the two strongest predictors of success rate of resolving WFD.

CONCLUSIONS

The Lexical Retrieval Hypothesis highlighting the facilitative functions of iconic and metaphoric gestures in lexical retrieval was not supported. Challenges in conducting research related to WFD, and the clinical implications in gesture-based language intervention for PWA were discussed.

Comprehensibility and neural substrate of communicative gestures in severe aphasia

Communicative gestures can compensate incomprehensibility of oral speech in severe aphasia, but the brain damage that causes aphasia may also have an impact on the production of gestures. We compared the comprehensibility of gestural communication of persons with severe aphasia and non-aphasic persons and used voxel based lesion symptom mapping (VLSM) to determine lesion sites that are responsible for poor gestural expression in aphasia. On group level, persons with aphasia conveyed more information via gestures than controls indicating a compensatory use of gestures in persons with severe aphasia. However, individual analysis showed a broad range of gestural comprehensibility. VLSM suggested that poor gestural expression was associated with lesions in anterior temporal and inferior frontal regions. We hypothesize that likely functional correlates of these localizations are selection of and flexible changes between communication channels as well as between different types of gestures and between features of actions and objects that are expressed by gestures.

Narratives of focal brain injured individuals: A macro-level analysis

Focal brain injury can have detrimental effects on the pragmatics of communication. This study examined narrative production by unilateral brain damaged people (n=36) and healthy controls and focused on the complexity (content and coherence) and the evaluative aspect of their narratives to test the general hypothesis that the left hemisphere is biased to process microlinguistic information and the right hemisphere is biased to process macrolinguistic information. We found that people with left hemisphere damage's (LHD) narratives were less likely to maintain the overall theme of the story and produced fewer evaluative comments in their narratives. These deficits correlated with their performances on microlinguistic linguistic tasks. People with the right hemisphere damage (RHD) seemed to be preserved in expressing narrative complexity and evaluations as a group. Yet, single case analyses revealed that particular regions in the right hemisphere such as damage to the dorsolateral prefrontal cortex (DLPFC), the anterior and superior temporal gyrus, the middle temporal gyrus, and the supramarginal gyrus lead to problems in creating narratives. Our findings demonstrate that both hemispheres are necessary to produce competent narrative production. LHD people's poor production is related to their microlinguistic language problems whereas RHD people's impaired abilities can be associated with planning and working memory abilities required to relate events in a narrative.

What gestures reveal about cognitive deficits in Williams Syndrome

Gestures have a central role in speaking and thinking about spatial information. The goal of the present study is to examine the function of gestures in Williams Syndrome (WS), a genetic disorder characterized by spatial impairments and a preservation of communication. The study's subjects were 11 WS individuals and 22 typically-developing controls who performed a narrative task. We analyzed offline gesture production and its relation with language and spatial information. Compared to the control groups, WS individuals produced more representational gestures that anticipated, supplemented, or gesture-only communication. Gestures produced by WS participants serve a compensatory role particularly in representing spatial contents.