Ideational Gestures and Speech in Brain-damaged Subjects

Gesture's Role in the Communication of Adults With Different Types of Aphasia

PURPOSE

Adults with aphasia gesture more than adults without aphasia. However, less is known about the role of gesture in different discourse contexts for individuals with different types of aphasia. In this study, we asked whether patterns of speech and gesture production of individuals with aphasia vary by aphasia and discourse type and also differ from the speech and gestures produced by adults without aphasia.

METHOD

We compared the amount, diversity, and complexity of speech and gesture production in adults with anomic or Broca's aphasia and adults with no aphasia (n = 20/group) in their first- versus third-person narratives.

RESULTS

Adults with Broca's aphasia showed the lowest performance in their amount, diversity, and complexity of speech production, followed by adults with anomic aphasia and adults without aphasia. This pattern was reversed for gesture production. Speech and gesture production also varied by discourse context. Adults with either type of aphasia used a lower amount of and less diverse speech in third-person than in first-person narratives; this pattern was also reversed for gesture production.

CONCLUSIONS

Overall, our results provide evidence for a compensatory role of gesture in aphasia communication. Adults with Broca's aphasia, who showed the greatest speech production difficulties, also relied most on gesture, and this pattern was particularly pronounced in the third-person narrative context.

Gestures and pauses to help thought: hands, voice, and silence in the tourist guide's speech

In the body of research on the relationship between gesture and speech, some models propose they form an integrated system while others attribute gestures a compensatory role in communication. This study addresses the gesture-speech relationship by taking disfluency phenomena as a case study. Since it is part of a project aimed at designing virtual agents to be employed in museums, an analysis was performed on the communicative behavior of tourist guides. Results reveal that gesturing is more frequent during speech than pauses. Moreover, when comparing the types of gestures and types of pauses they co-occur with, non-communicative gestures (idles and manipulators) turn out to be more frequent than communicatively-meaningful gestures, which instead more often co-occur with speech. We discuss these findings as relevant for a theoretical model viewing speech and gesture as an integrated system.

Can Gestures Give us Access to Thought? A Systematic Literature Review on the Role of Co-thought and Co-speech Gestures in Children with Intellectual Disabilities

This systematic review analyzes the differential use of gestures in learning by children with intellectual disability (ID) compared to typically developing ones (TD). Eleven studies published between 2000 and 2020 fulfilled the inclusion criteria (N = 364 participants). The results identify three key elements: (1) Children with ID accompany their spoken language with more gestures than TD children; (2) Specifically, they produce more iconic gestures that provide access to the conceptualization process and understanding in students with ID; (3) Children with ID rely on gesture more than TD children to carry meaning (i.e., produce unimodal gestural utterances without accompanying speech). Possible implications for teaching and guidelines for future research are proposed.

The Role of Gesture in Communication and Cognition: Implications for Understanding and Treating Neurogenic Communication Disorders

When people talk, they gesture. Gesture is a fundamental component of language that contributes meaningful and unique information to a spoken message and reflects the speaker's underlying knowledge and experiences. Theoretical perspectives of speech and gesture propose that they share a common conceptual origin and have a tightly integrated relationship, overlapping in time, meaning, and function to enrich the communicative context. We review a robust literature from the field of psychology documenting the benefits of gesture for communication for both speakers and listeners, as well as its important cognitive functions for organizing spoken language, and facilitating problem-solving, learning, and memory. Despite this evidence, gesture has been relatively understudied in populations with neurogenic communication disorders. While few studies have examined the rehabilitative potential of gesture in these populations, others have ignored gesture entirely or even discouraged its use. We review the literature characterizing gesture production and its role in intervention for people with aphasia, as well as describe the much sparser literature on gesture in cognitive communication disorders including right hemisphere damage, traumatic brain injury, and Alzheimer's disease. The neuroanatomical and behavioral profiles of these patient populations provide a unique opportunity to test theories of the relationship of speech and gesture and advance our understanding of their neural correlates. This review highlights several gaps in the field of communication disorders which may serve as a bridge for applying the psychological literature of gesture to the study of language disorders. Such future work would benefit from considering theoretical perspectives of gesture and using more rigorous and quantitative empirical methods in its approaches. We discuss implications for leveraging gesture to explore its untapped potential in understanding and rehabilitating neurogenic communication disorders.

The influence of conversation parameters on gesture production in aphasia

Conversation is important in everyday life and this importance is not diminished in aphasia. Context parameters such as topic and partner are known to influence the linguistic content of conversations. With gesture being closely linked to language, these parameters may influence gestures used in conversations. This has not been investigated in previous studies. This study explored the spontaneous use of gestures in the conversations of participants with aphasia (PWA) and neurologically healthy participants (NHP). It aimed to examine the influence of conversation topic and partner on gesture production overall and on the production of semantically rich and empty gestures. Twenty PWA and 21 NHP were filmed during conversations with different topics (narrative & procedural) and different partners (familiar & unfamiliar). Analysis 1 investigated the influence of the conversation topic on gesture production overall and on the production of semantically rich and empty gestures. In Analysis 2, the influence of the conversation partner on gesture production was investigated. Both groups produced significantly more gestures in procedural than in narrative conversations. Moreover, PWA and NHP produced significantly more semantically rich gestures in procedural conversations. In terms of the conversation partner, both groups produced significantly more gestures in the conversations with the unfamiliar than in those with the familiar conversation partner. For all findings, there were no group differences and no interactions between group and context parameters. These findings shed light on factors that influence gesture production and suggest that both modalities should be viewed together as a communicative resource for PWA.

The relationship between co-speech gesture production and macrolinguistic discourse abilities in people with focal brain injury

Brain damage is associated with linguistic deficits and might alter co-speech gesture production. Gesture production after focal brain injury has been mainly investigated with respect to intrasentential rather than discourse-level linguistic processing. In this study, we examined 1) spontaneous gesture production patterns of people with left hemisphere damage (LHD) or right hemisphere damage (RHD) in a narrative setting, 2) the neural structures associated with deviations in spontaneous gesture production in these groups, and 3) the relationship between spontaneous gesture production and discourse level linguistic processes (narrative complexity and evaluation competence). Individuals with LHD or RHD (17 people in each group) and neurotypical controls (n = 13) narrated a story from a picture book. Results showed that increase in gesture production for LHD individuals was associated with less complex narratives and lesions of individuals who produced more gestures than neurotypical individuals overlapped in frontal-temporal structures and basal ganglia. Co-speech gesture production of RHD individuals positively correlated with their evaluation competence in narrative. Lesions of RHD individuals who produced more gestures overlapped in the superior temporal gyrus and the inferior parietal lobule. Overall, LHD individuals produced more gestures than neurotypical individuals. The groups did not differ in their use of different gesture forms except that LHD individuals produced more deictic gestures per utterance than RHD individuals and controls. Our findings are consistent with the hypothesis that co-speech gesture production interacts with macro-linguistic levels of discourse and this interaction is affected by the hemispheric lateralization of discourse abilities.

Multimodal Communication in Aphasia: Perception and Production of Co-speech Gestures During Face-to-Face Conversation

The role of nonverbal communication in patients with post-stroke language impairment (aphasia) is not yet fully understood. This study investigated how aphasic patients perceive and produce co-speech gestures during face-to-face interaction, and whether distinct brain lesions would predict the frequency of spontaneous co-speech gesturing. For this purpose, we recorded samples of conversations in patients with aphasia and healthy participants. Gesture perception was assessed by means of a head-mounted eye-tracking system, and the produced co-speech gestures were coded according to a linguistic classification system. The main results are that meaning-laden gestures (e.g., iconic gestures representing object shapes) are more likely to attract visual attention than meaningless hand movements, and that patients with aphasia are more likely to fixate co-speech gestures overall than healthy participants. This implies that patients with aphasia may benefit from the multimodal information provided by co-speech gestures. On the level of co-speech gesture production, we found that patients with damage to the anterior part of the arcuate fasciculus showed a higher frequency of meaning-laden gestures. This area lies in close vicinity to the premotor cortex and is considered to be important for speech production. This may suggest that the use of meaning-laden gestures depends on the integrity of patients’ speech production abilities.

What gestures reveal about cognitive deficits in Williams Syndrome

Gestures have a central role in speaking and thinking about spatial information. The goal of the present study is to examine the function of gestures in Williams Syndrome (WS), a genetic disorder characterized by spatial impairments and a preservation of communication. The study's subjects were 11 WS individuals and 22 typically-developing controls who performed a narrative task. We analyzed offline gesture production and its relation with language and spatial information. Compared to the control groups, WS individuals produced more representational gestures that anticipated, supplemented, or gesture-only communication. Gestures produced by WS participants serve a compensatory role particularly in representing spatial contents.

Spontaneous gesture and spatial language: Evidence from focal brain injury

People often use spontaneous gestures when communicating spatial information. We investigated focal brain-injured individuals to test the hypotheses that (1) naming motion event components of manner-path (represented by verbs-prepositions in English) are impaired selectively, (2) gestures compensate for impaired naming. Patients with left or right hemisphere damage (LHD or RHD) and elderly control participants were asked to describe motion events (e.g., running across) depicted in brief videos. Damage to the left posterior middle frontal gyrus, left inferior frontal gyrus, and left anterior superior temporal gyrus (aSTG) produced impairments in naming paths of motion; lesions to the left caudate and adjacent white matter produced impairments in naming manners of motion. While the frequency of spontaneous gestures were low, lesions to the left aSTG significantly correlated with greater production of path gestures. These suggest that producing prepositions-verbs can be separately impaired and gesture production compensates for naming impairments when damage involves left aSTG.

Phonological similarity affects production of gestures, even in the absence of overt speech

Highlights

Does phonological similarity affect gesture production in the absence of speech?

Participants produced gestures from pictures with no words presented or spoken.

Same pictures and gestures but different training labels were used.

Phonologically similar labels led to more errors in subsequent gestures.

Thus, phonological similarity affects gesture production in the absence of speech.

Are manual gestures affected by inner speech? This study tested the hypothesis that phonological form influences gesture by investigating whether phonological similarity between words that describe motion gestures creates interference for production of those gestures in the absence of overt speech. Participants learned to respond to a picture of a bottle by gesturing to open the bottle's cap, and to a picture of long hair by gesturing to twirl the hair. In one condition, the gestures were introduced with phonologically-similar labels “twist” and “twirl” (similar condition), while in the other condition, they were introduced with phonologically-dissimilar labels “unscrew” and “twirl” (dissimilar condition). During the actual experiment, labels were not produced and participants only gestured by looking at pictures. In both conditions, participants also gestured to a control pair that was used as a baseline. Participants made significantly more errors on gestures in the similar than dissimilar condition after correction for baseline differences. This finding shows the influence of phonology on gesture production in the absence of overt speech and poses new constraints on the locus of the interaction between language and gesture systems.

Language and iconic gesture use in procedural discourse by speakers with aphasia

Background: Conveying instructions is an everyday use of language, and gestures are likely to be a key feature of this. Although co-speech iconic gestures are tightly integrated with language, and people with aphasia (PWA) produce procedural discourses impaired at a linguistic level, no previous studies have investigated how PWA use co-speech iconic gestures in these contexts. Aims: This study investigated how PWA communicated meaning using gesture and language in procedural discourses, compared with neurologically healthy people (NHP). We aimed to identify the relative relationship of gesture and speech, in the context of impaired language, both overall and in individual events. Methods & Procedures: Twenty-nine PWA and 29 NHP produced two procedural discourses. The structure and semantic content of language of the whole discourses were analysed through predicate argument structure and spatial motor terms, and gestures were analysed for frequency and semantic form. Gesture and language were analysed in two key events, to determine the relative information presented in each modality. Outcomes & Results: PWA and NHP used similar frequencies and forms of gestures, although PWA used syntactically simpler language and fewer spatial words. This meant, overall, relatively more information was present in PWA gesture. This finding was also reflected in the key events, where PWA used gestures conveying rich semantic information alongside semantically impoverished language more often than NHP. Conclusions: PWA gestures, containing semantic information omitted from the concurrent speech, may help listeners with meaning when language is impaired. This finding indicates gesture should be included in clinical assessments of meaning-making.

Iconic gesture in normal language and word searching conditions: a case of conduction aphasia

Although there is a substantive body of research about the language used by individuals with aphasia, relatively little is known about their spontaneous iconic gesture. A single case study of LT, an individual with conduction aphasia indicated qualitative differences between the spontaneous iconic gestures produced alongside fluent speech and in tip of the tongue states. The current study examined the iconic gestures produced by another individual with conduction aphasia, WT, and a group of 11 control participants. Comparisons were made between iconic gestures produced alongside normal language and those produced alongside word-searching behaviour. Participants recounted the Tweety and Sylvester cartoon Canary Row. All gesture produced was analysed qualitatively and quantitatively. WT produced more iconic gestures than controls accompanying word searching behaviour, whereas he produced a similar frequency of iconic gestures to control participants alongside normal language. The iconic gestures produced in the two language contexts also differed qualitatively. Frequency of iconic gesture production was not affected by limb apraxia. This study suggests that there are differences between iconic gestures that are produced alongside normal language and those produced alongside word-searching behaviour. Theoretical and clinical implications of these findings are discussed.

Naming and gesturing spatial relations: evidence from focal brain-injured individuals

Spatial language helps us to encode relations between objects and organize our thinking. Little is known about the neural instantiations of spatial language. Using voxel-lesion symptom mapping (VLSM), we tested the hypothesis that focal brain injured patients who had damage to left frontal-parietal peri-Sylvian regions would have difficulty in naming spatial relations between objects. We also investigated the relationship between impaired verbalization of spatial relations and spontaneous gesture production. Patients with left or right hemisphere damage and elderly control participants were asked to name static (e.g., an apple on a book) and dynamic (e.g., a pen moves over a box) locative relations depicted in brief video clips. The correct use of prepositions in each task and gestures that represent the spatial relations were coded. Damage to the left posterior middle frontal gyrus, the left inferior frontal gyrus, and the left anterior superior temporal gyrus were related to impairment in naming spatial relations. Production of spatial gestures negatively correlated with naming accuracy, suggesting that gestures might help or compensate for difficulty with lexical access. Additional analyses suggested that left hemisphere patients who had damage to the left posterior middle frontal gyrus and the left inferior frontal gyrus gestured less than expected, if gestures are used to compensate for impairments in retrieving prepositions.

Coverbal gestures in the recovery from severe fluent aphasia: a pilot study

This post hoc study investigated coverbal gesture patterns in two persons with chronic Wernicke's aphasia. They had both received therapy focusing on multimodal communication therapy, and their pre- and post-therapy verbal and gestural skills in face-to-face conversational interaction with their speech therapist were analysed by administering a partial barrier Referential Communication Task (RCT). The RCT sessions were reviewed in order to analyse: (a) participant coverbal gesture occurrence and types when in speaker role, (b) distribution of iconic gestures in the RCT communicative moves, (c) recognisable semantic content, and (d) the ways in which gestures were combined with empty or paraphasic speech. At post-therapy assessment only one participant showed improved communication skills in spite of his persistent language deficits. The improvement corresponded to changes on all gesturing measures, suggesting thereby that his communication relied more on gestural information. No measurable changes were observed for the non-responding participant-a finding indicating that the coverbal gesture measures used in this study might account for the different outcomes. These results point to the potential role of gestures in treatment aimed at fostering recovery from severe fluent aphasia. Moreover, this pattern of improvement runs contrary to a view of gestures used as a pure substitute for lexical items, in the communication of people with severe fluent aphasia.

LEARNING OUTCOMES

The readers will describe how to assess and interpret the patterns of coverbal gesturing in persons with fluent aphasia. They will also recognize the potential role of coverbal gestures in recovery from severe fluent aphasia.

Perceptions of communicative competence after traumatic brain injury: implications for ecologically-driven intervention targets

The present study investigated the relationship between non-verbal behaviours and perceptions of the communication abilities of an individual with anomia secondary to traumatic brain injury (TBI). Thirty-four university students studying Communication Sciences and Disorders were randomly assigned to watch or listen to six short clips of an individual with TBI engaged in conversation. Participants rated the individual on communication parameters from a modified version of the Pragmatic Protocol and four other dependent measures of communicative competence. A significant positive correlation was identified between perceptions of gestures and ratings of overall communicative competence, and between perceptions of hand and arm movements and ratings of overall communicative competence. Participant raters who viewed the individual's movements as inappropriate also rated her overall communication abilities less favourably. This finding highlights individuality in perception of communication competence and the importance of assessing communication partners' perceptions in a client's environment to determine socially relevant treatment goals.

Deictic word and gesture production: Their interaction

We examined whether and how deictic gestures and words influence each other when the content of the gesture was congruent or incongruent with that of the simultaneously produced word. Two experiments were carried out. In Experiment 1, the participants read aloud the deictic word 'QUA' ('here') or 'LA" ('there'), printed on a token placed near to or far from their body. Simultaneously, they pointed towards one's own body, when the token was placed near, or at a remote position, when the token was placed far. In this way, participants read 'QUA' ('here') and pointed towards themselves (congruent condition) or a remote position (incongruent condition); or they read 'LA" ('there') and pointed towards a remote position (congruent condition) or themselves (incongruent condition). In a control condition, in which a string of 'X' letters was printed on the token, the participants were silent and only pointed towards themselves (token placed near) or a remote position (token placed far). In Experiment 2, the participants read aloud the deictic word placed in the near or far position without gesturing. The results showed that the congruence/incongruence between the content of the deictic word and that of the gesture affected gesture kinematics and voice spectra. Indeed, the movement was faster in the congruent than in the control and incongruent conditions; and it was slower in the incongruent than in the control condition. As concerns voice spectra, formant 2 (F2) decreased in the incongruent conditions. The results suggest the existence of a bidirectional interaction between speech and gesture production systems.