Assimilation and contrast: the two sides of specific interference between action and perception

On the Role of Interoception in Body and Object Perception: A Multisensory-Integration Account

Various "embodied perception" phenomena suggest that what people sense of their body shapes what they perceive of the environment and that what they perceive of the environment shapes what they perceive of their bodies. For example, an observer's own hand can be felt where a fake hand is seen, events produced by own body movements seem to occur earlier than they did, and feeling a heavy weight at an observer's back may prompt hills to look steeper. Here we argue that such and various other phenomena are instances of multisensory integration of interoceptive signals from the body and exteroceptive signals from the environment. This overarching view provides a mechanistic description of what embodiment in perception means and how it works. It suggests new research questions while questioning a special role of the body itself and various phenomenon-specific explanations in terms of ownership, agency, or action-related scaling of visual information.

Shared Representations in Athletes: Segmenting Action Sequences From Taekwondo Reveals Implicit Agreement

How do athletes represent actions from their sport? How are these representations structured and which knowledge is shared among experts in the same discipline? To address these questions, the event segmentation task was used. Experts in Taekwondo and novices indicated how they would subjectively split videos of Taekwondo form sequences into meaningful units. In previous research, this procedure was shown to unveil the structure of internal action representations and to be affected by sensorimotor knowledge. Without specific instructions on the grain size of segmentation, experts tended to integrate over longer episodes which resulted in a lower number of single units. Moreover, in accordance with studies in figure-skating and basketball, we expected higher agreement among experts on where to place segmentation marks, i.e., boundaries. In line with this hypothesis, significantly more overlap of boundaries was found within the expert group as compared to the control group. This was observed even though the interindividual differences in the selected grain size were huge and expertise had no systematic influence here. The absence of obvious goals or objects to structure Taekwondo forms underlines the importance of shared expert knowledge. Further, experts might have benefited from sensorimotor skills which allowed to simulate the observed actions more precisely. Both aspects may explain stronger agreement among experts even in unfamiliar Taekwondo forms. These interpretations are descriptively supported by the participants’ statements about features which guided segmentation and by an overlap of the group’s agreed boundaries with those of an experienced referee. The study shows that action segmentation can be used to provide insights into structure and content of action representations specific to experts. The mechanisms underlying shared knowledge among Taekwondoists and among experts in general are discussed on the background of current theoretic frameworks.

Insula cortex gates the interplay of action observation and preparation for controlled imitation

Perceiving, anticipating and responding to the actions of another person are fundamentally entwined processes such that seeing another's movement can prompt automatic imitation, as in social mimicry and contagious yawning. Yet the direct-matching of others' movements is not always appropriate, so this tendency must be controlled. This necessitates the hierarchical integration of the systems for action mirroring with domain-general control networks. Here we use functional magnetic resonance imaging (fMRI) and computational modelling to examine the top-down and context-dependent modulation of mirror representations and their influence on motor planning. Participants performed actions that either intentionally or incidentally imitated, or counter-imitated, an observed action. Analyses of these fMRI data revealed a region in the mid-occipital gyrus (MOG) where activity differed between imitation versus counter-imitation in a manner that depended on whether this was intentional or incidental. To identify broader cortical network mechanisms underlying this interaction between intention and imitativeness, we used dynamic causal modelling to pose specific hypotheses which embody assumptions about inter-areal interactions and contextual modulations. These models each incorporated four regions - medial temporal V5 (early motion perception), MOG (action-observation), supplementary motor area (action planning), and anterior insula (executive control) - but differ in their interactions and hierarchical structure. The best model of our data afforded a crucial role for the anterior insula, gating the interaction of supplementary motor area and MOG activity. This provides a novel brain network-based account of task-dependent control over the integration of motor planning and mirror systems, with mirror responses suppressed for intentional counter-imitation.

Impact of proprioception on the perceived size and distance of external objects in a virtual action task

Previous research has revealed changes in the perception of objects due to changes of object-oriented actions. In present study, we varied the arm and finger postures in the context of a virtual reaching and grasping task and tested whether this manipulation can simultaneously affect the perceived size and distance of external objects. Participants manually controlled visual cursors, aiming at reaching and enclosing a distant target object, and judged the size and distance of this object. We observed that a visual-proprioceptive discrepancy introduced during the reaching part of the action simultaneously affected the judgments of target distance and of target size (Experiment 1). A related variation applied to the grasping part of the action affected the judgments of size, but not of distance of the target (Experiment 2). These results indicate that perceptual effects observed in the context of actions can directly arise through sensory integration of multimodal redundant signals and indirectly through perceptual constancy mechanisms.

Action affects perception through modulation of attention

The present study explored the origin of perceptual changes repeatedly observed in the context of actions. In Experiment 1, participants tried to hit a circular target with a stylus movement under restricted feedback conditions. We measured the perception of target size during action planning and observed larger estimates for larger movement distances. In Experiment 2, we then tested the hypothesis that this action specific influence on perception is due to changes in the allocation of spatial attention. For this purpose, we replaced the hitting task by conditions of focused and distributed attention and measured the perception of the former target stimulus. The results revealed changes in the perceived stimulus size very similar to those observed in Experiment 1. These results indicate that action’s effects on perception root in changes of spatial attention.

Event Perception and Memory

Events make up much of our lived experience, and the perceptual mechanisms that represent events in experience have pervasive effects on action control, language use, and remembering. Event representations in both perception and memory have rich internal structure and connections one to another, and both are heavily informed by knowledge accumulated from previous experiences. Event perception and memory have been identified with specific computational and neural mechanisms, which show protracted development in childhood and are affected by language use, expertise, and brain disorders and injuries. Current theoretical approaches focus on the mechanisms by which events are segmented from ongoing experience, and emphasize the common coding of events for perception, action, and memory. Abetted by developments in eye-tracking, neuroimaging, and computer science, research on event perception and memory is moving from small-scale laboratory analogs to the complexity of events in the wild.

The influence of motor preparation on the processing of action-relevant visual features

Action preparation can facilitate performance in tasks of visual perception, for instance by speeding up responses to action-relevant stimulus features. However, it is unknown whether this facilitation reflects an influence on early perceptual processing, or instead post-perceptual processes. In three experiments, a combination of psychophysics and electroencephalography was used to investigate whether visual features are influenced by action preparation at the perceptual level. Participants were cued to prepare oriented reach-to-grasp actions before discriminating target stimuli oriented in the same direction as the prepared grasping action (congruent) or not (incongruent). As expected, stimuli were discriminated faster if their orientation was congruent, compared to incongruent, with the prepared action. However, action-congruency had no influence on perceptual sensitivity, regardless of cue-target interval and discrimination difficulty. The reaction time effect was not accompanied by modulations of early visual-evoked potentials. Instead, beta-band (13-30 Hz) synchronization over sensorimotor brain regions was influenced by action preparation, indicative of improved response preparation. Together, the results suggest that action preparation may not modulate early visual processing of orientation, but likely influences higher order response or decision related processing. While early effects of action on spatial perception are well documented, separate mechanisms appear to govern non-spatial feature selection.

On perceptual biases in virtual object manipulation: Signal reliability and action relevance matter

This study examined the role of visual reliability and action relevance in mutual visual-proprioceptive attraction in a virtual grasping task. Participants initially enclosed either the width or the height of a visual rectangular object with two cursors controlled by the movements of the index finger and thumb. Then, either the height or the width of this object or the distance between the fingers was judged. The judgments of object's size were attracted by the felt finger distance, and, vice versa, the judged finger distance was attracted by the size of the grasped object. The impact of the proprioceptive information on object judgments increased, whereas the impact of visual object information on finger judgments decreased when the reliability of the visual stimulus was reduced. Moreover, the proprioceptive bias decreased for the action-relevant stimulus dimension as compared with the action-irrelevant stimulus dimension. These results indicate sensory integration of spatially separated sensory signals in the absence of any direct spatial or kinematic relation between them. We therefore suggest that the basic principles of sensory integration apply to the broad research field on perceptual-motor interactions as well as to many virtual interactions with external objects.

Visuomotor and motorvisual priming with different types of set-level congruency: evidence in support of ideomotor theory, and the planning and control model (PCM)

Perception can prime action (visuomotor priming), and action can prime perception (motorvisual priming). According to ideomotor theory both effects rely on the overlap of mental representations between perception and action. This implies that both effects get more pronounced the more features they share. We tested this hypothesis by employing in a motorvisual (Exp. 1) and in a visuomotor (Exp. 2) setting, three different pairs of left/right target stimuli (hand pictures, arrows, and words) varying in how strongly they overlap with the pair of left/right responses. For two stimulus pairs (hands and words) the hypothesis was confirmed: hand pictures share more features with the responses than words, consequently hand pictures produced a stronger visuomotor and a stronger motorvisual priming effect than words. However, arrow stimuli showed a different pattern: the temporal dynamics of both priming effects, as well as the direction of the effect seen in motorvisual priming, were significant but opposite to that of the hand and word stimuli. This suggests that the arrows' representations were not involved in ideomotor processes, and we propose instead that they were represented in a spatial or scalar fashion, outside the representations assumed in ideomotor theory. The results are discussed in the context of ideomotor theory, and the planning and control model of motorvisual priming.

Intentionally not imitating: Insula cortex engaged for top-down control of action mirroring

Perception and action are inextricably linked, down to the level of single cells which have both visual and motor response properties - dubbed 'mirror neurons'. The mirror neuron system is generally associated with direct-matching or resonance between observed and executed actions (and goals). Yet in everyday interactions responding to another's movements with matching actions (or goals) is not always appropriate. Here we examine processes associated with intentionally not imitating, as separable from merely detecting an observed action as mismatching one's own. Using fMRI, we test how matched and mismatched stimulus-response mapping for actions is modulated depending on task-relevance. Participants were either cued to intentionally copy or oppose a presented action (intentional imitation or counter-imitation), or cued to perform a predefined action regardless of the presented action (incidental imitation or counter-imitation). We found distinct cortical networks underlying imitation compared to counter-imitation, involving areas typically associated with an action observation network and widespread occipital activation. Intentionally counter-imitating particularly involved frontal-parietal networks, including the insula and cingulate cortices. This task-dependent recruitment of frontal networks for the intentional selection of opposing responses supports previous evidence for the preparatory suppression of imitative responses. Sensorimotor mirroring is modulated via control processes, which complex human interactions often require.

Testing the Motor Simulation Account of Source Errors for Actions in Recall

Observing someone else perform an action can lead to false memories of self-performance - the observation inflation effect. One explanation is that action simulation via mirror neuron activation during action observation is responsible for observation inflation by enriching memories of observed actions with motor representations. In three experiments we investigated this account of source memory failures, using a novel paradigm that minimized influences of verbalization and prior object knowledge. Participants worked in pairs to take turns acting out geometric shapes and letters. The next day, participants recalled either actions they had performed or those they had observed. Experiment 1 showed that participants falsely retrieved observed actions as self-performed, but also retrieved self-performed actions as observed. Experiment 2 showed that preventing participants from encoding observed actions motorically by taxing their motor system with a concurrent motor task did not lead to the predicted decrease in false claims of self-performance. Indeed, Experiment 3 showed that this was the case even if participants were asked to carefully monitor their recall. Because our data provide no evidence for a motor activation account, we also discussed our results in light of a source monitoring account.

Routes to embodiment

Research on embodiment is rich in impressive demonstrations but somewhat poor in comprehensive explanations. Although some moderators and driving mechanisms have been identified, a comprehensive conceptual account of how bodily states or dynamics influence behavior is still missing. Here, we attempt to integrate current knowledge by describing three basic psychological mechanisms: direct state induction, which influences how humans feel or process information, unmediated by any other cognitive mechanism; modal priming, which changes the accessibility of concepts associated with a bodily state; sensorimotor simulation, which affects the ease with which congruent and incongruent actions are performed. We argue that the joint impact of these mechanisms can account for most existing embodiment effects. Additionally, we summarize empirical tests for distinguishing these mechanisms and suggest a guideline for future research about the mechanisms underlying embodiment effects.

An action-incongruent secondary task modulates prediction accuracy in experienced performers: evidence for motor simulation

We provide behavioral evidence that the human motor system is involved in the perceptual decision processes of skilled performers, directly linking prediction accuracy to the (in)ability of the motor system to activate in a response-specific way. Experienced and non-experienced dart players were asked to predict, from temporally occluded video sequences, the landing position of a dart thrown previously by themselves (self) or another (other). This prediction task was performed while additionally performing (a) an action-incongruent secondary motor task (right arm force production), (b) a congruent secondary motor task (mimicking) or (c) an attention-matched task (tone-monitoring). Non-experienced dart players were not affected by any of the secondary task manipulations, relative to control conditions, yet prediction accuracy decreased for the experienced players when additionally performing the force-production, motor task. This interference effect was present for 'self' as well as 'other' decisions, reducing the accuracy of experienced participants to a novice level. The mimicking (congruent) secondary task condition did not interfere with (or facilitate) prediction accuracy for either group. We conclude that visual-motor experience moderates the process of decision making, such that a seemingly visual-cognitive prediction task relies on activation of the motor system for experienced performers. This fits with a motor simulation account of action prediction in sports and other tasks, and alerts to the specificity of these simulative processes.

Attentional resource allocation during a cued saccade task

Attentional selection of sensory information and motor output is critical for successful interaction with one's surroundings. However, organization of attentional processes involved in selection of salient visual information, decision making, and movement planning has not yet been fully elucidated. We hypothesized that attentional processes involved in these tasks can function independently and draw from separate resources. If true, challenging the capacity limit of one attentional process would not affect performance of others. Healthy participants performed a cued saccade task in which target cues were embedded in a central stream of letters in a Rapid Serial Visual Presentation (RSVP). Participants performed saccades as quickly and as accurately as possible to a peripheral target location based on cue presentation within the central letter stream. To challenge visual attention, we parametrically varied the duration at which each letter of the RSVP was presented (50-200ms). In a separate experiment we challenged motor attention by increasing the number of possible saccade trajectories (1-6 peripheral targets). As expected, increasing attentional load in one domain of the task negatively affected performance in that domain, while performance in other domains was unaffected. We interpret our results as support for the independent allocation of attentional resources, at least in the early stages of processing, required across components of a cued saccade task. Deciphering the contributions of attention during visuomotor tasks is a critical step to understanding how humans process information necessary to successfully interact with the environment.

Impact of planned movement direction on judgments of visual locations

The present study examined if and how the direction of planned hand movements affects the perceived direction of visual stimuli. In three experiments participants prepared hand movements that deviated regarding direction ("Experiment 1" and "2") or distance relative to a visual target position ("Experiment 3"). Before actual execution of the movement, the direction of the visual stimulus had to be estimated by means of a method of adjustment. The perception of stimulus direction was biased away from planned movement direction, such that with leftward movements stimuli appeared somewhat more rightward than with rightward movements. Control conditions revealed that this effect was neither a mere response bias, nor a result of processing or memorizing movement cues. Also, shifting the focus of attention toward a cued location in space was not sufficient to induce the perceptual bias observed under conditions of movement preparation ("Experiment 4"). These results confirm that characteristics of planned actions bias visual perception, with the direction of bias (contrast or assimilation) possibly depending on the type of the representations (categorical or metric) involved.

Investigating ideomotor cognition with motorvisual priming paradigms: key findings, methodological challenges, and future directions

Ideomotor theory claims that perceptual representations of action-effects are functionally involved in the planning of actions. Strong evidence for this claim comes from a phenomenon called motorvisual priming. Motorvisual priming refers to the finding that action planning directly affects perception, and that the effects are selective for stimuli that share features with the planned action. Motorvisual priming studies have provided detailed insights into the processing of perceptual representations in action planning. One important finding is that such representations in action planning have a categorical format, whereas metric representations are not anticipated in planning. Further essential findings regard the processing mechanisms and the time course of ideomotor cognition. Perceptual representations of action-effects are first activated by action planning and then bound into a compound representation of the action plan. This compound representation is stabilized throughout the course of the action by the shielding of all involved representations from other cognitive processes. Despite a rapid growth in the number of motorvisual priming studies in the current literature, there are still many aspects of ideomotor cognition which have not yet been investigated. These aspects include the scope of ideomotor processing with regard to action types and stimulus types, as well as the exact nature of the binding and shielding mechanisms involved.

Seeing while acting: hand movements can modulate attentional capture by motion onset

The onset of new motion has been shown to be a very robust cause of attentional capture, generating a processing advantage for the location of motion onset regardless of the observer's concurrent goal. The present study, motivated by the common-coding account of action and perception, examined whether the effect of motion onset on visual attention can be modulated by the observer's mode of action. Specifically, the common-coding account predicts that preparing an action can render the features that are used in the action plan less available for visual processes. Consistent with this hypothesis, in Experiment 1 the magnitude of attentional capture caused by a single motion onset was reduced when this motion was similar to the observer's response (i.e., along the same axis). Similarly, in Experiment 2 the onset of a response-different motion gained a processing advantage over the response-similar motion onset when the two were presented simultaneously. Since both types of motion were present in every trial, the results of Experiment 2 suggest that response similarity affected visual-attentional processes rather than motor processes. Together, these results suggest that the processes of attentional prioritization caused by motion onset can be modulated by the observer's concurrent action.

The planning and control model (PCM) of motorvisual priming: reconciling motorvisual impairment and facilitation effects

Previous research on dual-tasks has shown that, under some circumstances, actions impair the perception of action-consistent stimuli, whereas, under other conditions, actions facilitate the perception of action-consistent stimuli. We propose a new model to reconcile these contrasting findings. The planning and control model (PCM) of motorvisual priming proposes that action planning binds categorical representations of action features so that their availability for perceptual processing is inhibited. Thus, the perception of categorically action-consistent stimuli is impaired during action planning. Movement control processes, on the other hand, integrate multi-sensory spatial information about the movement and, therefore, facilitate perceptual processing of spatially movement-consistent stimuli. We show that the PCM is consistent with a wider range of empirical data than previous models on motorvisual priming. Furthermore, the model yields previously untested empirical predictions. We also discuss how the PCM relates to motorvisual research paradigms other than dual-tasks.