Developmental stages of perception and language acquisition in a perceptually grounded robot

Synthesizing the temporal self: robotic models of episodic and autobiographical memory

Episodic memories are experienced as belonging to a self that persists in time. We review evidence concerning the nature of human episodic memory and of the sense of self and how these emerge during development, proposing that the younger child experiences a persistent self that supports a subjective experience of remembering. We then explore recent research in cognitive architectures for robotics that has investigated the possibility of forms of synthetic episodic and autobiographical memory. We show that recent advances in generative modeling can support an understanding of the emergence of self and of episodic memory, and that cognitive architectures which include a language capacity are showing progress towards the construction of a narrative self with autobiographical memory capabilities for robots. We conclude by considering the prospects for a more complete model of mental time travel in robotics and the implications of this modeling work for understanding human episodic memory and the self in time. This article is part of the theme issue 'Elements of episodic memory: lessons from 40 years of research'.

The Role of Autobiographical Memory in the Development of a Robot Self

This article briefly reviews research in cognitive development concerning the nature of the human self. It then reviews research in developmental robotics that has attempted to retrace parts of the developmental trajectory of the self. This should be of interest to developmental psychologists, and researchers in developmental robotics. As a point of departure, one of the most characteristic aspects of human social interaction is cooperation-the process of entering into a joint enterprise to achieve a common goal. Fundamental to this ability to cooperate is the underlying ability to enter into, and engage in, a self-other relation. This suggests that if we intend for robots to cooperate with humans, then to some extent robots must engage in these self-other relations, and hence they must have some aspect of a self. Decades of research in human cognitive development indicate that the self is not fully present from the outset, but rather that it is developed in a usage-based fashion, that is, through engaging with the world, including the physical world and the social world of animate intentional agents. In an effort to characterize the self, Ulric Neisser noted that self is not unitary, and he thus proposed five types of self-knowledge that correspond to five distinct components of self: ecological, interpersonal, conceptual, temporally extended, and private. He emphasized the ecological nature of each of these levels, how they are developed through the engagement of the developing child with the physical and interpersonal worlds. Crucially, development of the self has been shown to rely on the child's autobiographical memory. From the developmental robotics perspective, this suggests that in principal it would be possible to develop certain aspects of self in a robot cognitive system where the robot is engaged in the physical and social world, equipped with an autobiographical memory system. We review a series of developmental robotics studies that make progress in this enterprise. We conclude with a summary of the properties that are required for the development of these different levels of self, and we identify topics for future research.

A boy named Sue

One major lesson learned in the cognitive sciences is that even basic human cognitive capacities are extraordinarily complicated and elusive to mechanistic explanations. This is definitely the case for naming and identity. Nothing seems simpler than using a proper name to refer to a unique individual object in the world. But psychological research has shown that the criteria and mechanisms by which humans establish and use names are unclear and seemingly contradictory. Children only develop the necessary knowledge and skills after years of development and naming degenerates in unusual selective ways with strokes, schizophrenia, or Alzheimer disease. Here we present an operational model of social interaction patterns and cognitive functions to explain how naming can be achieved and acquired. We study the Grounded Naming Game as a particular example of a symbolic interaction that requires naming and present mechanisms that build up and use the semiotic networks necessary for performance in the game. We demonstrate in experiments with autonomous physical robots that the proposed dynamical systems indeed lead to the formation of an effective naming system and that the model hence explains how naming and identity can get socially constructed and shared by a population of embodied agents.

An Alternative to Mapping a Word onto a Concept in Language Acquisition: Pragmatic Frames

The classic mapping metaphor posits that children learn a word by mapping it onto a concept of an object or event. However, we believe that a mapping metaphor cannot account for word learning, because even though children focus attention on objects, they do not necessarily remember the connection between the word and the referent unless it is framed pragmatically, that is, within a task. Our theoretical paper proposes an alternative mechanism for word learning. Our main premise is that word learning occurs as children accomplish a goal in cooperation with a partner. We follow Bruner's (1983) idea and further specify pragmatic frames as the learning units that drive language acquisition and cognitive development. These units consist of a sequence of actions and verbal behaviors that are co-constructed with a partner to achieve a joint goal. We elaborate on this alternative, offer some initial parametrizations of the concept, and embed it in current language learning approaches.

Exploring the acquisition and production of grammatical constructions through human-robot interaction with echo state networks

One of the principal functions of human language is to allow people to coordinate joint action. This includes the description of events, requests for action, and their organization in time. A crucial component of language acquisition is learning the grammatical structures that allow the expression of such complex meaning related to physical events. The current research investigates the learning of grammatical constructions and their temporal organization in the context of human-robot physical interaction with the embodied sensorimotor humanoid platform, the iCub. We demonstrate three noteworthy phenomena. First, a recurrent network model is used in conjunction with this robotic platform to learn the mappings between grammatical forms and predicate-argument representations of meanings related to events, and the robot's execution of these events in time. Second, this learning mechanism functions in the inverse sense, i.e., in a language production mode, where rather than executing commanded actions, the robot will describe the results of human generated actions. Finally, we collect data from naïve subjects who interact with the robot via spoken language, and demonstrate significant learning and generalization results. This allows us to conclude that such a neural language learning system not only helps to characterize and understand some aspects of human language acquisition, but also that it can be useful in adaptive human-robot interaction.

Learning to attend: a connectionist model of situated language comprehension

Evidence from numerous studies using the visual world paradigm has revealed both that spoken language can rapidly guide attention in a related visual scene and that scene information can immediately influence comprehension processes. These findings motivated the coordinated interplay account (Knoeferle & Crocker, 2006) of situated comprehension, which claims that utterance-mediated attention crucially underlies this closely coordinated interaction of language and scene processing. We present a recurrent sigma-pi neural network that models the rapid use of scene information, exploiting an utterance-mediated attentional mechanism that directly instantiates the CIA. The model is shown to achieve high levels of performance (both with and without scene contexts), while also exhibiting hallmark behaviors of situated comprehension, such as incremental processing, anticipation of appropriate role fillers, as well as the immediate use, and priority, of depicted event information through the coordinated use of utterance-mediated attention to the scene.

A Neurolinguistic Model of Grammatical Construction Processing

One of the functions of everyday human language is to communicate meaning. Thus, when one hears or reads the sentence, “John gave a book to Mary,” some aspect of an event concerning the transfer of possession of a book from John to Mary is (hopefully) transmitted. One theoretical approach to language referred to as construction grammar emphasizes this link between sentence structure and meaning in the form of grammatical constructions. The objective of the current research is to (1) outline a functional description of grammatical construction processing based on principles of psycholinguistics, (2) develop a model of how these functions can be implemented in human neurophysiology, and then (3) demonstrate the feasibility of the resulting model in processing languages of typologically diverse natures, that is, English, French, and Japanese. In this context, particular interest will be directed toward the processing of novel compositional structure of relative phrases. The simulation results are discussed in the context of recent neurophysiological studies of language processing.