Maternal Interaction Relates to Neural Processing of Self-Related Multisensory Information in 5-Month-Olds

The ontogenetic origin of the self in infancy is a topic of ongoing debate. Although influential developmental and neurocognitive theories propose that caregiver-infant interactions play an important role in infants' self-development, little is known about the specific mechanisms involved. Some theories highlight the importance of caregiver sensitivity and touch, while others propose that caregiver contingency plays a central role. The study aimed to investigate infants' self-perception by measuring brain activation in the posterior superior temporal sulcus (pSTS), a region previously associated with self-related processing. A total of 118 mother-infant dyads participated in a free-play interaction, during which maternal sensitivity and touch were measured. Additionally, a face-to-face interaction was conducted to measure maternal contingency. Infants' brain activation was measured using functional near-infrared spectroscopy (fNIRS). They watched a video of their own face while being stroked by a brush on the cheek. The video was either live and the stroking was synchronous to the video (contingent) or the video was delayed by 3 s, which made the stroking asynchronous (non-contingent). The results showed that infants exhibited more HbO-activation in the right pSTS in the non-contingent condition. Importantly, the more sensitive the mothers were and the more they touched infants during free play, the less differential activation the infants showed in response to both conditions. This effect was driven by infants showing less activation to the non-contingent condition when their mothers exhibited more care, maybe because of a smaller prediction error for non-contingent self-related multisensory information. Overall, the study deepens our knowledge of how early social interactions relate to the emergence of the self in infancy.

After a Hand Was Lent: Sporadically Experiencing Multisensory Interference During the Rubber Hand Illusion Does Not Shield Against Disembodiment

Observations from multisensory body illusions indicate that the body representation can be adapted to changing task demands, e.g., it can be expanded to integrate external objects based on current sensorimotor experience (embodiment). While the mechanisms that promote embodiment have been studied extensively in earlier work, the opposite phenomenon of, removing an embodied entity from the body representation (i.e., disembodiment) has received little attention yet. The current study addressed this phenomenon and drew inspiration from the partial reinforcement extinction effect in instrumental learning which suggests that behavior is more resistant to extinction when reinforcement is delivered irregularly. In analogy to this, we investigated whether experiencing occasional visuo-motor mismatches during the induction phase of the moving rubber hand illusion (intermittent condition) would result in slower disembodiment as compared to a regular induction phase where motor and visual signals always match (continuous condition). However, we did not find an effect of reinforcement schedule on disembodiment. Keeping a recently embodied entity in the body schema, therefore, requires constant updating through correlated perceptual and motor signals.

Linking actions and memories: Probing the interplay of action-effect congruency, agency experience, and recognition memory

Adult humans experience agency when their action causes certain events (sense of agency). Moreover, they can later remember what these events were (memory). Here, we investigate how the relationship between actions and events shapes agency experience and memory for the corresponding events. Participants performed actions that produced stimuli that were either congruent or incongruent to the action while memory of these stimuli was probed in a recognition test. Additionally, predictability of the effect was manipulated in Experiment 1 by using either randomly interleaved or blocked ordering of action-congruent and action-incongruent events. In Experiment 2, the size of the action space was manipulated by allowing participants to choose between three or six possible responses. The results indicated a heightened sense of agency following congruent compared to incongruent trials, with this effect being increased given a larger available action space, as well as a greater sense of agency given higher predictability of the effect. Recognition memory was better for stimuli presented in congruent compared to incongruent trials, with no discernible effects of effect predictability or the size of the action space. The results point towards a joint influence of predictive and postdictive processes on agency experience and suggest a link between control and memory. The partial dissociation of influences on agency experience and memory cast doubt on a mediating role of agency experience on the relationship between action-effect congruency and memory. Theoretical accounts for this relationship are discussed.

Social exclusion in a virtual Cyberball game reduces the virtual hand illusion

Sense of ownership and agency are two important aspects of the minimal self, but how self-perception is affected by social conditions remains unclear. Here, we studied how social inclusion or exclusion of participants in the course of a virtual Cyberball game would affect explicit judgments and implicit measures of ownership and agency (proprioceptive drift, skin conductance responses, and intentional binding, respectively) in a virtual hand illusion paradigm, in which a virtual hand moved in or out of sync with the participants' own hand. Results show that synchrony affected all four measures. More importantly, this effect interacted with social inclusion/exclusion in the Cyberball game for both ownership and agency measure, showing that social exclusion reduces perceived agency and ownership.

Freedom of choice boosts midfrontal theta power during affective feedback processing of goal-directed actions

Sense of agency, the feeling of being in control of one's actions and their effects, is particularly relevant during goal-directed actions. During feedback learning, action effects provide information about the best course of action to reinforce positive and prevent negative outcomes. However, it is unclear whether agency experience selectively affects the processing of negative or positive feedback during the performance of goal-directed actions. As an important marker of feedback processing, we examined agency-related changes in midfrontal oscillatory activity in response to performance feedback using electroencephalography. Thirty-three participants completed a reinforcement learning task during which they received positive (monetary gain) or negative (monetary loss) feedback following item choices made either by themselves (free-choice) or by the computer (forced-choice). Independent of choice context, midfrontal theta activity was more enhanced for negative than positive feedback. In addition, free, compared to forced choices increased midfrontal theta power for both gain and loss feedback. These results indicate that freedom of choice in a motivationally salient learning task leads to a general enhancement in the processing of affective action outcomes. Our findings contribute to an understanding of the neuronal mechanisms underlying agency-related changes during action regulation and indicate midfrontal theta activity as a neurophysiological marker important for the monitoring of affective action outcomes, irrespective of feedback valence.

GOALIATH: a theory of goal-directed behavior

Commonsense and theorizing about action control agree in assuming that human behavior is (mainly) driven by goals, but no mechanistic theory of what goals are, where they come from, and how they impact action selection is available. Here I develop such a theory that is based on the assumption that GOALs guide Intentional Actions THrough criteria (GOALIATH). The theory is intended to be minimalist and parsimonious with respect to its assumptions, as transparent and mechanistic as possible, and it is based on representational assumptions provided by the Theory of Event Coding (TEC). It holds that goal-directed behavior is guided by selection criteria that activate and create competition between event files that contain action-effect codes matching one or more of the criteria-a competition that eventually settles into a solution favoring the best-matching event file. The criteria are associated with various sources, including biological drives, acquired needs (e.g., of achievement, power, or affiliation), and short-term, sometimes arbitrary, instructed aims. Action selection is, thus, a compromise that tries to satisfy various criteria related to different driving forces, which are also likely to vary in strength over time. Hence, what looks like goal-directed action emerges from, and represents an attempt to satisfy multiple constraints with different origins, purposes, operational characteristics, and timescales-which among other things does not guarantee a high degree of coherence or rationality of the eventual outcome. GOALIATH calls for a radical break with conventional theorizing about the control of goal-directed behavior, as it among other things questions existing cognitive-control theories and dual-route models of action control.

Phenomenological Origins of Psychological Ownership

Motivated by a set of converging empirical findings and theoretical suggestions pertaining to the construct of ownership, we survey literature from multiple disciplines and present an extensive theoretical account linking the inception of a foundational naïve theory of ownership to principles governing the sense of (body) ownership. The first part of the account examines the emergence of the non-conceptual sense of ownership in terms of the minimal self and the body schema—a dynamic mental model of the body that functions as an instrument of directed action. A remarkable feature of the body schema is that it expands to incorporate objects that are objectively controlled by the person. Moreover, this embodiment of extracorporeal objects is accompanied by the phenomenological feeling of ownership towards the embodied objects. In fact, we argue that the sense of agency and ownership are inextricably linked, and that predictable control over an object can engender the sense of ownership. This relation between objective agency and the sense of ownership is moderated by gestalt-like principles. In the second part, we posit that these early emerging principles and experiences lead to the formation of a naïve theory of ownership rooted in notions of agential involvement.

Perspective Taking and Avatar-Self Merging

Today, avatars often represent users in digital worlds such as in video games or workplace applications. Avatars embody the user and perform their actions in these artificial environments. As a result, users sometimes develop the feeling that their self merges with their avatar. The user realizes that they are the avatar, but the avatar is also the user-meaning that avatar's appearance, character, and actions also affect their self. In the present paper, we first introduce the event-coding approach of the self and then argue based on the reviewed literature on human-avatar interaction that a self-controlled avatar can lead to avatar-self merging: the user sets their own goals in the virtual environment, plans and executes the avatar's actions, and compares the predicted with the actual motion outcomes of the avatar. This makes the user feel body ownership and agency over the avatar's action. Following the event-coding account, avatar-self merging should not be seen as an all-or-nothing process, but rather as a continuous process to which various factors contribute, including successfully taking the perspective of the avatar. Against this background, we discuss affective, cognitive, and visuo-spatial perspective taking of the avatar. As evidence for avatar-self merging, we present findings showing that when users take the avatar's perspective, they can show spontaneous behavioral tendencies that run counter to their own.

Environment-Related and Body-Related Components of the Minimal Self

Perceptual changes that an agent produces by efferent activity can become part of the agent’s minimal self. Yet, in human agents, efferent activities produce perceptual changes in various sensory modalities and in various temporal and spatial proximities. Some of these changes occur at the “biological” body, and they are to some extent conveyed by “private” sensory signals, whereas other changes occur in the environment of that biological body and are conveyed by “public” sensory signals. We discuss commonalties and differences of these signals for generating selfhood. We argue that despite considerable functional overlap of these sensory signals in generating self-experience, there are reasons to tell them apart in theorizing and empirical research about development of the self.

An Embodied Cognition Perspective on the Role of Interoception in the Development of the Minimal Self

Interoception is an often neglected but crucial aspect of the human minimal self. In this perspective, we extend the embodiment account of interoceptive inference to explain the development of the minimal self in humans. To do so, we first provide a comparative overview of the central accounts addressing the link between interoception and the minimal self. Grounding our arguments on the embodiment framework, we propose a bidirectional relationship between motor and interoceptive states, which jointly contribute to the development of the minimal self. We present empirical findings on interoception in development and discuss the role of interoception in the development of the minimal self. Moreover, we make theoretical predictions that can be tested in future experiments. Our goal is to provide a comprehensive view on the mechanisms underlying the minimal self by explaining the role of interoception in the development of the minimal self.

The auditory brain in action: Intention determines predictive processing in the auditory system-A review of current paradigms and findings

According to the ideomotor theory, action may serve to produce desired sensory outcomes. Perception has been widely described in terms of sensory predictions arising due to top-down input from higher order cortical areas. Here, we demonstrate that the action intention results in reliable top-down predictions that modulate the auditory brain responses. We bring together several lines of research, including sensory attenuation, active oddball, and action-related omission studies: Together, the results suggest that the intention-based predictions modulate several steps in the sound processing hierarchy, from preattentive to evaluation-related processes, also when controlling for additional prediction sources (i.e., sound regularity). We propose an integrative theoretical framework—the extended auditory event representation system (AERS), a model compatible with the ideomotor theory, theory of event coding, and predictive coding. Initially introduced to describe regularity-based auditory predictions, we argue that the extended AERS explains the effects of action intention on auditory processing while additionally allowing studying the differences and commonalities between intention- and regularity-based predictions—we thus believe that this framework could guide future research on action and perception.

The Me-File: An Event-Coding Approach to Self-Representation

Numerous authors have taken it for granted that people represent themselves or even have something like “a self”, but the underlying mechanisms remain a mystery. How do people represent themselves? Here I propose that they do so not any differently from how they represent other individuals, events, and objects: by binding codes representing the sensory consequences of being oneself into a Me-File, that is, into an event file integrating all the codes resulting from the behaving me. This amounts to a Humean bundle-self theory of selfhood, and I will explain how recent extensions of the Theory of Event Coding, a general theory of human perception and action control, provide all the necessary ingredients for specifying the mechanisms underlying such a theory. The Me-File concept is likely to provide a useful mechanistic basis for more specific and more theoretically productive experimentation, as well as for the construction of artificial agents with human-like selves.

How Action Shapes Body Ownership Momentarily and Throughout the Lifespan

Objects which a human agent controls by efferent activities (such as real or virtual tools) can be perceived by the agent as belonging to his or her body. This suggests that what an agent counts as “body” is plastic, depending on what she or he controls. Yet there are possible limitations for such momentary plasticity. One of these limitations is that sensations stemming from the body (e.g., proprioception) and sensations stemming from objects outside the body (e.g., vision) are not integrated if they do not sufficiently “match”. What “matches” and what does not is conceivably determined by long–term experience with the perceptual changes that body movements typically produce. Children have accumulated less sensorimotor experience than adults have. Consequently, they express higher flexibility to integrate body-internal and body-external signals, independent of their “match” as suggested by rubber hand illusion studies. However, children’s motor performance in tool use is more affected by mismatching body-internal and body-external action effects than that of adults, possibly because of less developed means to overcome such mismatches. We review research on perception-action interactions, multisensory integration, and developmental psychology to build bridges between these research fields. By doing so, we account for the flexibility of the sense of body ownership for actively controlled events and its development through ontogeny. This gives us the opportunity to validate the suggested mechanisms for generating ownership by investigating their effects in still developing and incomplete stages in children. We suggest testable predictions for future studies investigating both body ownership and motor skills throughout the lifespan.

Computational models of the "active self" and its disturbances in schizophrenia

The notion that self-disorders are at the root of the emergence of schizophrenia rather than a symptom of the disease, is getting more traction in the cognitive sciences. This is in line with philosophical approaches that consider an enactive self, constituted through action and interaction with the environment. We thereby analyze different definitions of the self and evaluate various computational theories lending to these ideas. Bayesian and predictive processing are promising approaches for computational modeling of the "active self". We evaluate their implementation and challenges in computational psychiatry and cognitive developmental robotics. We describe how and why embodied robotic systems provide a valuable tool in psychiatry to assess, validate, and simulate mechanisms of self-disorders. Specifically, mechanisms involving sensorimotor learning, prediction, and self-other distinction, can be assessed with artificial agents. This link can provide essential insights to the formation of the self and new avenues in the treatment of psychiatric disorders.

Sensorimotor Representation Learning for an “Active Self” in Robots: A Model Survey

Safe human-robot interactions require robots to be able to learn how to behave appropriately in spaces populated by people and thus to cope with the challenges posed by our dynamic and unstructured environment, rather than being provided a rigid set of rules for operations. In humans, these capabilities are thought to be related to our ability to perceive our body in space, sensing the location of our limbs during movement, being aware of other objects and agents, and controlling our body parts to interact with them intentionally. Toward the next generation of robots with bio-inspired capacities, in this paper, we first review the developmental processes of underlying mechanisms of these abilities: The sensory representations of body schema, peripersonal space, and the active self in humans. Second, we provide a survey of robotics models of these sensory representations and robotics models of the self; and we compare these models with the human counterparts. Finally, we analyze what is missing from these robotics models and propose a theoretical computational framework, which aims to allow the emergence of the sense of self in artificial agents by developing sensory representations through self-exploration.

Prerequisites for an Artificial Self

Traditionally investigated in philosophy, body ownership and agency-two main components of the minimal self-have recently gained attention from other disciplines, such as brain, cognitive and behavioral sciences, and even robotics and artificial intelligence. In robotics, intuitive human interaction in natural and dynamic environments becomes more and more important, and requires skills such as self-other distinction and an understanding of agency effects. In a previous review article, we investigated studies on mechanisms for the development of motor and cognitive skills in robots (Schillaci et al., 2016). In this review article, we argue that these mechanisms also build the foundation for an understanding of an artificial self. In particular, we look at developmental processes of the minimal self in biological systems, transfer principles of those to the development of an artificial self, and suggest metrics for agency and body ownership in an artificial self.

Sensorimotor Contingencies as a Key Drive of Development: From Babies to Robots

Much current work in robotics focuses on the development of robots capable of autonomous unsupervised learning. An essential prerequisite for such learning to be possible is that the agent should be sensitive to the link between its actions and the consequences of its actions, called sensorimotor contingencies. This sensitivity, and more particularly its role as a key drive of development, has been widely studied by developmental psychologists. However, the results of these studies may not necessarily be accessible or intelligible to roboticians. In this paper, we review the main experimental data demonstrating the role of sensitivity to sensorimotor contingencies in infants' acquisition of four fundamental motor and cognitive abilities: body knowledge, memory, generalization, and goal-directedness. We relate this data from developmental psychology to work in robotics, highlighting the links between these two domains of research. In the last part of the article we present a blueprint architecture demonstrating how exploitation of sensitivity to sensorimotor contingencies, combined with the notion of "goal," allows an agent to develop new sensorimotor skills. This architecture can be used to guide the design of specific computational models, and also to possibly envisage new empirical experiments.

Theory of Event Coding (TEC) V2.0: Representing and controlling perception and action

This article provides an update of the Theory of Event Coding (TEC), which claims that perception and action are identical processes operating on the same codes - event files consisting of integrated networks of sensorimotor feature codes. The original version of the theory emphasized its representational underpinnings, but recent theoretical developments provide the basis for a more integrated view consisting of both the codes that are shared between perception and action in the control processes operating on these codes. Four developments are discussed in more detail: The degree to which the integration and retrieval of event files depends on current goals, how metacontrol states impact the handling of event files, how feature binding relates to event learning, and how the integration of non-social events relates to the integration of social events. Case examples using various versions of the Simon task are used to explain how the new version of TEC explains interactions between perception and action in non-social and social situations.

Context-induced contrast and assimilation effects in explicit and implicit measures of agency

Virtual-hand-illusion studies often use explicit and implicit measures of body ownership but no agreed-on implicit measure of agency exists. We investigated whether the Intentional Binding (IB) effect could serve as such a measure. A pilot study confirmed that current consistency increases both perceived agency and IB. In three experiments, current consistency was 50% but the previously experienced consistency was either 100% or 0%. When previous and present consistency experience were separated by a short break, both explicit judgments and IB showed a contrast effect. Eliminating the break reversed the effect in explicit agency but not in IB; and making the transition between previous and present consistency smoother replicated the effect for explicit agency but reversed the pattern for IB. Our findings suggest that explicit agency and IB rely on different sources of information, presumably including cross-sensory correlations, predictions of expected action-effects, and comparisons between present and previous consistency experiences.

Representing Oneself and Others

Human beings are assumed to own a concept of their self, but it remains a mystery how they represent themselves and others. I shall develop a theoretical framework, inspired by the Theory of Event Coding, of how people represent themselves and others, how and under which circumstances these two kinds of representations interact and what consequences this has. In a nutshell, I shall argue that self- and other-representations can overlap to the degree that they share features, that the shared features are particularly relevant or salient, and that the individual is under a particular metacontrol state. Then I shall argue that self-concepts emerge through active exploration of one's physical and social environment during infancy and childhood, as well as through cultural learning, and that their main purpose is related to social communication but not online action control.

From anticipation to integration: the role of integrated action-effects in building sensorimotor contingencies

Ideomotor approaches to action control have provided evidence that the activation of an anticipatory image of previously learned action-effects plays a decisive role in action selection. This study sought for converging evidence by combining three previous experimental paradigms: the response-effect compatibility protocol introduced by Kunde (Journal of Experimental Psychology: Human Perception and Performance, 27(2), 387-394, 2001), the acquisition-test paradigm developed by Elsner and Hommel (Journal of Experimental Psychology: Human Perception and Performance, 27(1), 229, 2001), and the object-action compatibility manipulation of Tucker and Ellis (Visual Cognition, 8(6), 769-800, 2001). Three groups of participants first performed a response-effect compatibility task, in which they carried out power and precision grasps that produced either grasp-compatible or grasp-incompatible pictures, or no action effects. Performance was better in the compatible than in the incompatible group, which replicates previous observations and extends them to relationships between grasps and objects. Then, participants were to categorize object pictures by carrying out grasp responses. Apart from replicating previous findings of better performance in trials in which object size and grasp type was compatible, we found that this stimulus-response compatibility effect depended on previous response-effect learning. Taken together, these findings support the assumption that the experience of action-effect contingencies establishes durable event files that integrate representations of actions and their effects.