A conceptual framework for the study of demonstrative reference

Spatial communication systems across languages reflect universal action constraints

The extent to which languages share properties reflecting the non-linguistic constraints of the speakers who speak them is key to the debate regarding the relationship between language and cognition. A critical case is spatial communication, where it has been argued that semantic universals should exist, if anywhere. Here, using an experimental paradigm able to separate variation within a language from variation between languages, we tested the use of spatial demonstratives-the most fundamental and frequent spatial terms across languages. In n = 874 speakers across 29 languages, we show that speakers of all tested languages use spatial demonstratives as a function of being able to reach or act on an object being referred to. In some languages, the position of the addressee is also relevant in selecting between demonstrative forms. Commonalities and differences across languages in spatial communication can be understood in terms of universal constraints on action shaping spatial language and cognition.

What's your point? Insights from virtual reality on the relation between intention and action in the production of pointing gestures

Human communication involves the process of translating intentions into communicative actions. But how exactly do our intentions surface in the visible communicative behavior we display? Here we focus on pointing gestures, a fundamental building block of everyday communication, and investigate whether and how different types of underlying intent modulate the kinematics of the pointing hand and the brain activity preceding the gestural movement. In a dynamic virtual reality environment, participants pointed at a referent to either share attention with their addressee, inform their addressee, or get their addressee to perform an action. Behaviorally, it was observed that these different underlying intentions modulated how long participants kept their arm and finger still, both prior to starting the movement and when keeping their pointing hand in apex position. In early planning stages, a neurophysiological distinction was observed between a gesture that is used to share attitudes and knowledge with another person versus a gesture that mainly uses that person as a means to perform an action. Together, these findings suggest that our intentions influence our actions from the earliest neurophysiological planning stages to the kinematic endpoint of the movement itself.

Learning to generate pointing gestures in situated embodied conversational agents

One of the main goals of robotics and intelligent agent research is to enable them to communicate with humans in physically situated settings. Human communication consists of both verbal and non-verbal modes. Recent studies in enabling communication for intelligent agents have focused on verbal modes, i.e., language and speech. However, in a situated setting the non-verbal mode is crucial for an agent to adapt flexible communication strategies. In this work, we focus on learning to generate non-verbal communicative expressions in situated embodied interactive agents. Specifically, we show that an agent can learn pointing gestures in a physically simulated environment through a combination of imitation and reinforcement learning that achieves high motion naturalness and high referential accuracy. We compared our proposed system against several baselines in both subjective and objective evaluations. The subjective evaluation is done in a virtual reality setting where an embodied referential game is played between the user and the agent in a shared 3D space, a setup that fully assesses the communicative capabilities of the generated gestures. The evaluations show that our model achieves a higher level of referential accuracy and motion naturalness compared to a state-of-the-art supervised learning motion synthesis model, showing the promise of our proposed system that combines imitation and reinforcement learning for generating communicative gestures. Additionally, our system is robust in a physically-simulated environment thus has the potential of being applied to robots.

Choice of Mandarin Spatial Demonstratives in Distant Interaction

Scholars are divided on whether the Speaker's (S') choice of spatial demonstratives in verbal interaction is ego-centric or not. We studied the choice of "zhe (here)/ na (there)" by a Mandarin S instructing Hearer (H) from a few to dozens of meters. Using within-group and between-group experiments in a picture-description paradigm, we found that both S- and H- distance (Ds and Dh) to the Referent constantly influence S' demonstrative choice, and the social relation as a variable (Relation) between S and H also exerts some influence. Our findings support the idea that spatial reference in verbal interaction is somewhat non-egocentric.

Demonstrative systems: From linguistic typology to social cognition

This study explores the connection between language and social cognition by empirically testing different typological analyses of various demonstrative systems. Linguistic typology classifies demonstrative systems as distance-oriented or person-oriented, depending on whether they indicate the location of a referent relative only to the speaker, or to both the speaker and the listener. From the perspective of social cognition, speakers of languages with person-oriented systems must monitor their listener's spatial location in order to accurately use their demonstratives, while speakers of languages with distance-oriented systems can use demonstratives from their own, egocentric perspective. Resolving an ongoing controversy around the nature of the Spanish demonstrative system, the results of Experiment 1 confirmed that this demonstrative system is person oriented, while the English system is distance oriented. Experiment 2 revealed that not all three-way demonstrative systems are person oriented, with Japanese speakers showing sensitivity to the listener's spatial location, while Turkish speakers did not show such an effect in their demonstrative choice. In Experiment 3, Catalan-Spanish bilinguals showed sensitivity to listener position in their choice of the Spanish distal form, but not in their choice of the medial form. These results were interpreted as a transfer effect from Catalan, which revealed analogous results to English. Experiment 4 investigated the use of demonstratives to redirect a listener's attention to the intended referent, which is a universal function of demonstratives that also hinges on social cognition. Japanese and Spanish speakers chose between their proximal and distal demonstratives flexibly, depending on whether the listener was looking closer or further from the referent, whereas Turkish speakers chose their medial form for attention correction. In conclusion, the results of this study support the view that investigating how speakers of different languages jointly use language and social cognition in communication has the potential to unravel the deep connection between these two fundamentally human capacities.

Learning speaker- and addressee-centered demonstratives in Ticuna

This study investigated the acquisition of demonstratives (e.g., this/that, here/there) by 45 children (1;0 - 4;11) learning Ticuna, an Indigenous Amazonian language with an unusually complex demonstrative system. I analyzed 89 10-minute samples from video recordings of child-caregiver interaction, examining how often children and caregivers produced each demonstrative type, as well as relationships among children's age, children's demonstrative production, and caregivers' production. Caregivers' demonstrative production displayed few relationships with children's age or production. Children produced speaker-proximal and speaker-distal demonstratives (this near me, that far from me) earlier in developmental time than addressee-proximal demonstratives (that near you), and nominal (this/that) demonstratives earlier than locative (here/there) ones. Compared to caregivers, children overused speaker-proximal demonstratives, but used other demonstrative types with adult-like frequency beginning at 2;0. These results support the view that cognitive biases toward egocentric, proximal, and semantically simpler items substantially influence children's acquisition of demonstratives.

Shrinking Your Deictic System: How Far Can You Go?

Languages around the world differ in terms of the number of adnominal and pronominal demonstratives they require, as well as the factors that impact on their felicitous use. Given this cross-linguistic variation in deictic demonstrative terms, and the features that determine their felicitous use, an open question is how this is accommodated within bilingual cognition and language. In particular, we were interested in the extent to which bilingual language exposure and practice might alter the way in which a bilingual is using deictic demonstratives in their first language. Recent research on language attrition suggests that L2 learning selectively affects aspects of the native language, with some domains of language competence being more vulnerable than others. If demonstratives are basic, and acquired relatively early, they should be less susceptible to change and attrition. This was the hypothesis we went on to test in the current study. We tested two groups of native Spanish speakers, a control group living in Spain and an experimental group living in Norway using the (Spatial) Memory game paradigm. Contra to our expectations, the results indicate a significant difference between the two groups in use of deictic terms, indicative of a change in the preferred number of terms used. This suggests that deictic referential systems may change over time under pressure from bilingual language exposure.