Gesture Helps, Only If You Need It: Inhibiting Gesture Reduces Tip-of-the-Tongue Resolution for Those With Weak Short-Term Memory

Gesture production at encoding supports narrative recall

Existing research is inconsistent regarding the effects of gesture production on narrative recall. Most studies have examined the effects of gesture production during a recall phase, not during encoding, and findings regarding gesture's effects are mixed. The present study examined whether producing gestures at encoding could benefit an individual's narrative recall and whether this effect is moderated by verbal memory and spatial ability. This study also investigated whether producing certain types of gesture is most beneficial to recalling details of a narrative. Participants read a narrative aloud while producing their own gestures at pre-specified phrases in the narrative (Instructed Gesture condition), while placing both their hands behind their backs (No Gesture condition) or with no specific instructions regarding gesture (Spontaneous Gesture condition). Participants completed measures of spatial ability and verbal memory. Recall was measured through both free recall, and specific recall questions related to particular phrases in the narrative. Spontaneous gesture production at encoding benefited free recall, while instructed gestures provided the greatest benefit for recall of specific phrases where gesture had been prompted during encoding. Conversely, for recall of specific phrases where gesture had not been prompted during encoding, instructions to either gesture or not gesture suppressed recall for those higher in verbal memory. Finally, producing iconic and deictic gestures provided benefits for narrative recall, whilst beat gestures had no effect. Gestures play an important role in how we encode and subsequently recall information, providing an opportunity to support cognitive capacity.

Human-Machine Interaction via Dual Modes of Voice and Gesture Enabled by Triboelectric Nanogenerator and Machine Learning

With the development of science and technology, human-machine interaction has brought great benefits to the society. Here, we design a voice and gesture signal translator (VGST), which can translate natural actions into electrical signals and realize efficient communication in human-machine interface. By spraying silk protein on the copper of the device, the VGST can achieve improved output and a wide frequency response of 20-2000 Hz with a high sensitivity of 167 mV/dB, and the resolution of frequency detection can reach 0.1 Hz. By designing its internal structure, its resonant frequency and output voltage can be adjusted. The VGST can be used as a high-fidelity platform to effectively recover recorded music and can also be combined with machine learning algorithms to realize the function of speech recognition with a high accuracy rate of 97%. It also has good antinoise performance to recognize speech correctly even in noisy environments. Meanwhile, in gesture recognition, the triboelectric translator is able to recognize simple hand gestures and to judge the distance between hand and the VGST based on the principle of electrostatic induction. This work demonstrates that triboelectric nanogenerator (TENG) technology can have great application prospects and significant advantages in human-machine interaction and high-fidelity platforms.

Gestures and pauses to help thought: hands, voice, and silence in the tourist guide's speech

In the body of research on the relationship between gesture and speech, some models propose they form an integrated system while others attribute gestures a compensatory role in communication. This study addresses the gesture-speech relationship by taking disfluency phenomena as a case study. Since it is part of a project aimed at designing virtual agents to be employed in museums, an analysis was performed on the communicative behavior of tourist guides. Results reveal that gesturing is more frequent during speech than pauses. Moreover, when comparing the types of gestures and types of pauses they co-occur with, non-communicative gestures (idles and manipulators) turn out to be more frequent than communicatively-meaningful gestures, which instead more often co-occur with speech. We discuss these findings as relevant for a theoretical model viewing speech and gesture as an integrated system.

Over-reliance on English hinders cognitive science

English is the dominant language in the study of human cognition and behavior: the individuals studied by cognitive scientists, as well as most of the scientists themselves, are frequently English speakers. However, English differs from other languages in ways that have consequences for the whole of the cognitive sciences, reaching far beyond the study of language itself. Here, we review an emerging body of evidence that highlights how the particular characteristics of English and the linguistic habits of English speakers bias the field by both warping research programs (e.g., overemphasizing features and mechanisms present in English over others) and overgeneralizing observations from English speakers' behaviors, brains, and cognition to our entire species. We propose mitigating strategies that could help avoid some of these pitfalls.

A Hands-On Approach to Learning: Gesture Production During Encoding and its Effect on Narrative Recall

Research has shown that gesture production supports learning across a number of tasks. It is unclear, however, whether gesture production during encoding can support narrative recall, who gesture production benefits most, and whether certain types of gestures are more beneficial than others. This study, therefore, investigated the effect of gesture production during the encoding of a narrative on subsequent narrative recall, and whether individuals' levels of verbal and nonverbal memory moderated this effect. Additionally, this study investigated whether producing certain types of gestures during encoding was more beneficial than others. Participants (N = 90, M_age = 20.43) read aloud a narrative while under instruction to produce gestures, under no specific instruction to produce gestures, or were required to keep their hands behind their back to prevent them from gesturing. Participants completed measures assessing verbal and nonverbal memory. While gesture production during encoding benefitted narrative recall (as measured through specific questions), verbal memory moderated the effect, such that gesture production was more beneficial for individuals with higher than lower verbal memory. Furthermore, producing representational gestures during encoding benefitted recall of points in the narrative at which those gestures were produced, while beat gestures had no effect. Findings have implications for understanding the mechanisms underlying the links between gesture and learning, as well as practical implications in instructional settings.

The Effect of Simulating Climbing Movements on Rock Memory and Exploratory Movement in Rock Climbing

Simulating climbing movements on a given route is important for fluent rock climbing. We investigated the effect of simulated action during rock climbing route finding on memory and exploratory movement. Participants were 12 climbers and 12 non-climbers who completed three experimental tasks: (a) a questionnaire, the Vividness of Movement Imagery Questionnaire-2 (VMIQ-2) for measuring vividness of motor imagery, (b) a memory task requiring recognition of rock climbing holds on the route, and (c) a traversing task requiring participants to climb a given route and count the number of exploratory movements made during climbing. During route finding in the memory/traversing task, we experimentally manipulated the simulative body action with motor imagery. Results showed that the simulative action affected exploratory movement during climbing, but it did not affect memorization of the holds. In the traversing task, climbers showed more exploratory movement when the simulative action was present during route finding, while the non-climbers showed an opposite trend. Moreover, for non-climbers, the effect of the simulative action was modulated by the vividness of kinesthetic imagery. We concluded that simulative body action during route finding facilitated motor imagery and spatial information processing for subsequent climbing involving exploratory movement.

Socially Shared Feelings of Imminent Recall: More Tip-of-the-Tongue States Are Experienced in Small Groups

Tip-of-the-tongue (TOT) states are typically defined as feelings of imminent recall for known, but temporarily inaccessible target words. However, TOTs are not merely instances of retrieval failures. Clues that increase the subjective likelihood of retrieval success, such as cue familiarity and target-related information, also have been shown to elicit feelings of imminent recall, supporting a metacognitive, inferential etiology of the TOT phenomenon. A survey conducted on our university campus provided anecdotal evidence that TOTs are occasionally shared among people in small groups. Although shared TOTs may suggest the influence of social contagion, we hypothesized that metacognitive appraisal of group recall efficiency could be involved. There should be more instances of remembering in several heads than in one. From this, we conjectured that people remembering together entertain the inference that successful retrieval is more likely in group recall than in a single-person recall situation. Such a metacognitive appraisal may drive a stronger feeling of closeness with the target word and of recall imminence, precipitating one (or more people) into a TOT state. We used general knowledge questions to elicit TOTs. We found that participants reported more TOTs when remembering in small groups than participants remembering alone. Critically, the experimental manipulation selectively increased TOTs without affecting correct recall, suggesting that additional TOTs observed in small groups were triggered independently from the retrieval process. Near one third (31%) of the TOTs in small groups were reported by two or more participants for the same items. However, removing common TOTs from the analyses did not change the basic pattern of results, suggesting that social contagion was not the main factor involved in the observed effect. We argue that beyond social contagion, group recall magnifies the inference that target words will be successfully retrieved, prompting the metacognitive monitoring system to launch more near-retrieval success "warning" (TOT) signals than in a single-person recall situation.

The Relation Between Cognitive Abilities and the Distribution of Semantic Features Across Speech and Gesture in 4-year-olds

When young children learn to use language, they start to use their hands in co-verbal gesturing. There are, however, considerable differences between children, and it is not completely understood what these individual differences are due to. We studied how children at 4 years of age employ speech and iconic gestures to convey meaning in different kinds of spatial event descriptions, and how this relates to their cognitive abilities. Focusing on spontaneous illustrations of actions, we applied a semantic feature (SF) analysis to characterize combinations of speech and gesture meaning and related them to the child's visual-spatial abilities or abstract/concrete reasoning abilities (measured using the standardized SON-R 2 1 2 - 7 test). Results show that children with higher cognitive abilities convey significantly more meaning via gesture and less solely via speech. These findings suggest that young children's use of cospeech representational gesturing is positively related to their mental representation and reasoning abilities.