Conceptual effects on representational momentum

Representational horizon and visual space orientation: An investigation into the role of visual contextual cues on spatial mislocalisations

The perceived offset position of a moving target has been found to be displaced forward, in the direction of motion (Representational Momentum; RM), downward, in the direction of gravity (Representational Gravity; RG), and, recently, further displaced along the horizon implied by the visual context (Representational Horizon; RH). The latter, while still underexplored, offers the prospect to clarify the role of visual contextual cues in spatial orientation and in the perception of dynamic events. As such, the present work sets forth to ascertain the robustness of Representational Horizon across varying types of visual contexts, particularly between interior and exterior scenes, and to clarify to what degree it reflects a perceptual or response phenomenon. To that end, participants were shown targets, moving along one out of several possible trajectories, overlaid on a randomly chosen background depicting either an interior or exterior scene rotated -22.5º, 0º, or 22.5º in relation to the actual vertical. Upon the vanishing of the target, participants were required to indicate its last seen location with a computer mouse. For half the participants, the background vanished with the target while for the remaining it was kept visible until a response was provided. Spatial localisations were subjected to a discrete Fourier decomposition procedure to obtain independent estimates of RM, RG, and RH. Outcomes showed that RH's direction was biased towards the horizon implied by the visual context, but solely for exterior scenes, and irrespective of its presence or absence during the spatial localisation response, supporting its perceptual/representational nature.

Motion perception in touch: resolving contradictory findings by varying probabilities of different trial types

Representational momentum describes the typical overestimation of the final location of a moving stimulus in the direction of stimulus motion. While systematically observed in different sensory modalities, especially vision and audition, in touch, empirical findings indicate a mixed pattern of results, with some published studies suggesting the existence of the phenomenon, while others do not. In the present study, one possible moderating variable, the relative probabilities of different trial types, was explored in an attempt to resolve the seemingly contradictory findings in the literature. In some studies, only consistently moving target stimuli were presented and no representational momentum was observed, while other studies have included inconsistently moving target stimuli in the same experimental block, and observed representational momentum. Therefore, the present study was designed to systematically compare the localization of consistent target motion stimuli across two experimental blocks, for which either only consistent motion trials were presented, or else mixed with inconsistent target motion trials. The results indicate a strong influence of variations in the probability of different trial types on the occurrence of representational momentum. That is, representational momentum only occurred when both trial types (inconsistent and consistent target motion) were presented within one experimental block. The results are discussed in light of recent theoretical advancements in the literature, namely the speed prior account of motion perception.

Need for (expected) speed: Exploring the indirect influence of trial type consistency on representational momentum

The biases affecting people's perception of dynamic stimuli are typically robust and strong for specific stimulus configurations. For example, representational momentum describes a systematic perceptual bias in the direction of motion for the final location of a moving stimulus. Under clearly defined stimulus configurations (e.g., specific stimulus identity, size, speed), for example, the frequently used "implied motion" trial sequence, for which a target is subsequently presented in a consistent direction and with a consistent speed, a displacement in motion direction is evidenced. The present study explores the potential influence of expectations regarding directional as well as speed consistencies on representational momentum, elicited by including other, inconsistently moving trial types within the same experimental block. A systematic representational momentum effect was observed when only consistent motion trials were presented. In contrast, when inconsistent target motion trials were mixed within the same block of experimental trials, the representational momentum effect decreased, or was even eliminated (Experiments 1 & 2). Detailed analysis indicated that this reflects a global (proportion of consistent and inconsistent motion trials within a particular experimental block), not local (preceding trial influencing actual trial) effect. Yet, additional follow-up studies (Experiments 3 & 4) support the idea that these changes in perceived location are strongly influenced by the overall stimulus speed statistics in the different experimental blocks. These results are discussed and interpreted in light of recent theoretical developments in the literature on motion perception that highlight the importance of expectations about stimulus speed for motion perception.

Serial dependence and representational momentum in single-trial perceptual decisions

The human brain has evolved to predict and anticipate environmental events from their temporal dynamics. Predictions can bias perception toward the recent past, particularly when the environment contains no foreseeable changes, but can also push perception toward future states of sensory input, like when anticipating the trajectory of moving objects. Here, we show that perceptual decisions are simultaneously influenced by both past and future states of sensory signals. Using an orientation adjustment task, we demonstrate that single-trial errors are displaced toward previous features of behaviorally relevant stimuli and, at the same time, toward future states of dynamic sensory signals. These opposing tendencies, consistent with decisional serial dependence and representational momentum, involve different types of processing: serial dependence occurs beyond objecthood whereas representational momentum requires the representation of a single object with coherent dynamics in time and space. The coexistence of these two phenomena supports the independent binding of stimuli and decisions over time.

Static internal representation of dynamic situations reveals time compaction in human cognition

Introduction

The human brain has evolved under the constraint of survival in complex dynamic situations. It makes fast and reliable decisions based on internal representations of the environment. Whereas neural mechanisms involved in the internal representation of space are becoming known, entire spatiotemporal cognition remains a challenge. Growing experimental evidence suggests that brain mechanisms devoted to spatial cognition may also participate in spatiotemporal information processing.

Objectives

The time compaction hypothesis postulates that the brain represents both static and dynamic situations as purely static maps. Such an internal reduction of the external complexity allows humans to process time-changing situations in real-time efficiently. According to time compaction, there may be a deep inner similarity between the representation of conventional static and dynamic visual stimuli. Here, we test the hypothesis and report the first experimental evidence of time compaction in humans.

Methods

We engaged human subjects in a discrimination-learning task consisting in the classification of static and dynamic visual stimuli. When there was a hidden correspondence between static and dynamic stimuli due to time compaction, the learning performance was expected to be modulated. We studied such a modulation experimentally and by a computational model.

Results

The collected data validated the predicted learning modulation and confirmed that time compaction is a salient cognitive strategy adopted by the human brain to process time-changing situations. Mathematical modelling supported the finding. We also revealed that men are more prone to exploit time compaction in accordance with the context of the hypothesis as a cognitive basis for survival.

Conclusions

The static internal representation of dynamic situations is a human cognitive mechanism involved in decision-making and strategy planning to cope with time-changing environments. The finding opens a new venue to understand how humans efficiently interact with our dynamic world and thrive in nature.

Predictions from masked motion with and without obstacles

Predicting the future is essential for organisms like Homo sapiens, who live in a dynamic and ever-changing world. Previous research has established that conscious stimuli can lead to non-conscious predictions. Here we examine whether masked stimuli can also induce such predictions. We use masked movement-with and without obstacles-to examine predictions from masked stimuli. In six experiments a moving object was masked using continuous flash suppression (CFS). A few hundred milliseconds after the object had disappeared, a conscious probe appeared in a location that was either consistent with the masked stimulus or not. In Experiments 1-3 the movement was linear, and reaction times (RTs) indicated predictions that were based on direction and speed of movement. In Experiment 4, the masked moving object collided with an obstacle and then disappeared. Predictions in this case should reflect deflection, and indeed reaction times revealed predictions on the deflection route. In Experiments 5 and 6 we introduce an innovative way of using eye-tracking during continuous flash suppression (CFS) and report physiological evidence-in the forms of eye-movements-for masked stimuli induced predictions. We thus conclude that humans can use dynamic masked stimuli to generate active predictions about the future, and use these predictions to guide behavior. We also discuss the possible interpretations of these findings in light of the current scientific discussion regarding the relation between masked presentation, subliminal perception and awareness measurement methods.

Probing the effect of the expected-speed violation illusion

Motion perception is complex for the brain to process, involving interacting computations of distance, time, and speed. These computations can be biased by the context and the features of the perceived moving object, giving rise to several types of motion illusions. Recent research has shown that, in addition to object features and context, lifelong priors can bias attributes of perception. In the present work, we investigated if such long acquired expectations can bias speed perception. Using a two-interval forced-choice (2-IFC) task, we asked 160 participants in different experiments to judge which of two vehicles, one archetypically fast (e.g. a motorbike), and one comparatively slower (e.g. a bike), was faster. By varying the objective speeds of the two-vehicle types, and measuring the participants' point of subjective equality, we observed a consistent bias in participants' speed perception. Counterintuitively, in the first three experiments the speed of an archetypically slow vehicle had to be decreased relative to that of an archetypically fast vehicle, for the two to be judged as the same. Similarly, in the next three experiments, an archetypically fast vehicle's speed had to be increased relative to an archetypically slow vehicle's speed, for the two to be perceived as equal. Four additional control experiments replicated our results. We define this newly found bias as the expected-speed violation illusion (ESVI). We believe the ESVI as conceptually very similar to the size-weight illusion, and discuss it within the Bayesian framework of human perception.

The space contraction asymmetry in Michotte's launching effect

Previous studies have found that, compared with noncausal events, spatial contraction exists between the causal object and the effect object due to the perceived causality. The present research aims to examine whether the causal object and the effect object have the same effect on spatial contraction. A modified launching effect, in which a bar bridges the spatial gap between the final position of the launcher and the initial position of the target, was adopted. Experiment 1 validates the absolute underestimation of the bar's length between the launcher and the target. Experiment 2a finds that in the direct launching effect, the perceived position of the bar's trailing edge that was contacted by the final launcher was displaced along the objects' direction of movement. Meanwhile, the perceived position of the bar's leading edge that was contacted by the initial target was displaced in opposite direction to the moving direction. The magnitude of the former's displacement was significantly larger than that of the latter, displaying a significant contraction asymmetry. Experiment 2b demonstrates that the contraction asymmetry did not result from the launcher remaining in contact with the edge of the bar. Experiment 3 indicates that contraction asymmetry showed a type of postdictive effect; that is, to some extent, this asymmetry depends on what happens after contact. In conclusion, the space between the causal object and effect object contracts asymmetrically in the launching effect, which implies that the causal object and effect object are perceived as shifting toward each other nonequidistantly in visual space.

The role of cortical areas hMT/V5+ and TPJ on the magnitude of representational momentum and representational gravity: a transcranial magnetic stimulation study

The perceived vanishing location of a moving target is systematically displaced forward, in the direction of motion-representational momentum-, and downward, in the direction of gravity-representational gravity. Despite a wealth of research on the factors that modulate these phenomena, little is known regarding their neurophysiological substrates. The present experiment aims to explore which role is played by cortical areas hMT/V5+, linked to the processing of visual motion, and TPJ, thought to support the functioning of an internal model of gravity, in modulating both effects. Participants were required to perform a standard spatial localization task while the activity of the right hMT/V5+ or TPJ sites was selectively disrupted with an offline continuous theta-burst stimulation (cTBS) protocol, interspersed with control blocks with no stimulation. Eye movements were recorded during all spatial localizations. Results revealed an increase in representational gravity contingent on the disruption of the activity of hMT/V5+ and, conversely, some evidence suggested a bigger representational momentum when TPJ was stimulated. Furthermore, stimulation of hMT/V5+ led to a decreased ocular overshoot and to a time-dependent downward drift of gaze location. These outcomes suggest that a reciprocal balance between perceived kinematics and anticipated dynamics might modulate these spatial localization responses, compatible with a push-pull mechanism.

The effect of symbolic meaning of speed on time to contact

The effects of moving task-irrelevant objects on time-to-contact (TTC) judgments are examined in six experiments. In particular, we investigated the effects of the symbolic meaning of speed on TTC by presenting images of objects recalling the symbolic meaning of high speed (motorbike, rocket, formula one, rabbit, cheetah and flying Superman) and low speed (bicycle, hot-air balloon, tank, turtle, elephant and static Superman). In all experiments, participants judged the TTC of these moving objects with a black line, indicating the end of the occlusion. Experiment 7 was conducted to disambiguate whether the effects on TTC, found in the previous experiments, were either a by-product of a speed illusion or they were rather elicited by the implicit timing task. In a two-interval forced choice task, participants were instructed to judge if "high-speed objects" moved actually faster than "slow-speed objects". The results revealed no consistent speed illusion. Taken together the results showed shorter TTC estimated with stimuli recalling the meaning of high compared to low speed, but only with the long occlusion duration (3.14 s). At shorter occlusion durations, the pattern was reversed (participant tend to have shorter TTC with stimuli recalling the meaning of low speed). We suggest that the symbolic meaning of speed works mainly at low speed and long TTC, because the semantic elaboration of the stimulus needs a deeper cognitive elaboration. On the other hand, at higher speeds, a small erroneous perceptual judgment affects the TTC, perhaps due to a speed expectancy violation of the expected "slow object".

Arousing emoticons edit stream/bounce perception of objects moving past each other

When two identical objects move toward each other, overlap completely, and continue toward opposite ends of a space, observers might perceive them as streaming through or bouncing off each other. This phenomenon is known as ‘stream/bounce perception’. In this study, we investigated the effect of the presentation of emoticons on stream/bounce perception in five experiments. In Experiment 1, we used emoticons representing anger (‘(‘∧’)’), a smile (‘(^_^)’), and a sober face (‘(°_°)’, as a control), and observers were asked to judge whether two objects unrelated to the emoticon had streamed through or bounced off each other. The anger emoticon biased perception toward bouncing when compared with the smile or sober face emoticon. In Experiments 2 and 3, we controlled for the valence and arousal of emoticons, and found that arousal influenced stream/bounce perception but valence did not. Experiments 4 and 5 ruled out the possibility of attentional capture and response bias for the emoticon with higher arousal. Taken together, the findings indicate that emoticons with higher arousal evoke a mental image of a ‘collision’ in observers, thereby eliciting the bounce perception.

Toward a general theory of momentum-like effects

The future actions, behaviors, and outcomes of objects, individuals, and processes can often be anticipated, and some of these anticipations have been hypothesized to result from momentum-like effects. Five types of momentum-like effects (representational momentum, operational momentum, attentional momentum, behavioral momentum, psychological momentum) are briefly described. Potential similarities involving properties of momentum-like effects (continuation, coherence, role of chance or guessing, role of sensory processing, imperviousness to practice or error feedback, shifts in memory for position, effects of changes in velocity, rapid occurrence, effects of retention interval, attachment to an object rather than an abstract frame of reference, nonrigid transformation) are described, and potential constraints on a future theory of momentum-like effects (dynamic representation, nature of extrapolation, sensitivity to environmental contingencies, bridging gaps between stimulus and response, increasing adaptiveness to the environment, serving as a heuristic for perception and action, insensitivity to stimulus format, importance of subjective consequences, role of knowledge and belief, automaticity of occurrence, properties of functional architecture) are discussed. The similarity and ubiquity of momentum-like effects suggests such effects might result from a single or small number of mechanisms that operate over different dimensions, modalities, and time-scales and provide a fundamental adaptation for perception and action.

Vestibular stimulation interferes with the dynamics of an internal representation of gravity

The remembered vanishing location of a moving target has been found to be displaced downward in the direction of gravity (representational gravity) and more so with increasing retention intervals, suggesting that the visual spatial updating recruits an internal model of gravity. Despite being consistently linked with gravity, few inquiries have been made about the role of vestibular information in these trends. Previous experiments with static tilting of observers' bodies suggest that under conflicting cues between the idiotropic vector and vestibular signals, the dynamic drift in memory is reduced to a constant displacement along the body's main axis. The present experiment aims to replicate and extend these outcomes while keeping the observers' bodies unchanged in relation to physical gravity by varying the gravito-inertial acceleration using a short-radius centrifuge. Observers were shown, while accelerated to varying degrees, targets moving along several directions and were required to indicate the perceived vanishing location after a variable interval. Increases of the gravito-inertial force (up to 1.4G), orthogonal to the idiotropic vector, did not affect the direction of representational gravity, but significantly disrupted its time course. The role and functioning of an internal model of gravity for spatial perception and orientation are discussed in light of the results.

Representational momentum reveals visual anticipation differences in the upper and lower visual fields

Recent empirical research has revealed differences in functional capacity between the upper and lower visual fields (VFs), with the lower VF exhibiting superiority in visual perception skills. Similarly, functional differences between the left and right hemispheres elicit a predominance for visuospatial processing in the left visual field (left VF). Both anatomical as well as evolutionary arguments have been adopted in accounting for these variations in function. Preceding upper and lower VF research has typically investigated either static stimulus perception or the visual processing of upper limb action. The aim of the current research was to investigate whether the lower VF benefits associated with limb control transcend to visual anticipation (the perception of motion). Methods were based on Khan and Lawrence (Exp Brain Res 164:395-398, 2005), who investigated upper/lower VF differences in visuomotor control, but utilising a representational momentum paradigm to isolate perceptual processes. Thirty-two participants were randomised into either a left or right VF group and completed a perceptual computer-based task in the upper and lower VF, where they were required to judge the final position of a moving object before it disappeared. Two aspects of the distributions of same responses were then analysed; the central tendency (weighted means) and the variability. Results revealed that in the left VF, weighted means for the lower VF were significantly greater than for the upper VF [t(14) = 2.242, p = 0.042]. In both left and right VFs, variability was greater in the upper compared to lower VF. This provides new findings regarding visual processes in the different visual fields. While visual search and large scene perception has been found to be superior in the upper VF, here we find that visual anticipation, like target-directed visuomotor skill, is superior in the lower VF.

Perceived displacement explains wolfpack effect

We investigate the influence of perceived displacement of moving agent-like stimuli on the performance in dynamic interactive tasks. In order to reliably measure perceived displacement we utilize multiple tasks with different task demands. The perceived center of an agent's body is displaced in the direction in which the agent is facing and this perceived displacement is larger than the theoretical position of the center of mass would predict. Furthermore, the displacement in the explicit judgment is dissociated from the displacement obtained by the implicit measures. By manipulating the location of the pivot point, we show that it is not necessary to postulate orientation as an additional cue utilized by perception, as has been suggested by earlier studies. These studies showed that the agent's orientation influences the detection of chasing motion and the detection-related performance in interactive tasks. This influence has been labeled wolfpack effect. In one of the demonstrations of the wolfpack effect participants control a green circle on a display with a computer mouse. It has been shown that participants avoid display areas with agents pointing toward the green circle. Participants do so in favor of areas where the agents point in the direction perpendicular to the circle. We show that this avoidance behavior arises because the agent's pivot point selected by the earlier studies is different from where people locate the center of agent's body. As a consequence, the nominal rotation confounds rotation and translation. We show that the avoidance behavior disappears once the pivot point is set to the center of agent's body.

An effect of contrast and luminance on visual representational momentum for location

Effects of the contrast of target luminance and background luminance, and of the absolute level of target luminance, on representational momentum for the remembered final location of a previously viewed moving target were examined. Targets were high in contrast or luminance, decreasing in contrast or luminance, increasing in contrast or luminance, or low in contrast or luminance; the background was black or white. Representational momentum for target location was larger if targets were high or increasing in contrast or luminance and smaller if targets were low or decreasing in contrast or luminance. Representational momentum for target location was larger if targets were presented on a white background than on a black background. Implications for theories of localization and for theories of representational momentum are discussed.

High-level context effects on spatial displacement: the effects of body orientation and language on memory

Three decades of research suggests that cognitive simulation of motion is involved in the comprehension of object location, bodily configuration, and linguistic meaning. For example, the remembered location of an object associated with actual or implied motion is typically displaced in the direction of motion. In this paper, two experiments explore context effects in spatial displacement. They provide a novel approach to estimating the remembered location of an implied motion image by employing a cursor-positioning task. Both experiments examine how the remembered spatial location of a person is influenced by subtle differences in implied motion, specifically, by shifting the orientation of the person's body to face upward or downward, and by pairing the image with motion language that differed on intentionality, fell versus jumped. The results of Experiment 1, a survey-based experiment, suggest that language and body orientation influenced vertical spatial displacement. Results of Experiment 2, a task that used Adobe Flash and Amazon Mechanical Turk, showed consistent effects of body orientation on vertical spatial displacement but no effect of language. Our findings are in line with previous work on spatial displacement that uses a cursor-positioning task with implied motion stimuli. We discuss how different ways of simulating motion can influence spatial memory.

Multiple routes to mental animation: language and functional relations drive motion processing for static images

When looking at static visual images, people often exhibit mental animation, anticipating visual events that have not yet happened. But what determines when mental animation occurs? Measuring mental animation using localized brain function (visual motion processing in the middle temporal and middle superior temporal areas, MT+), we demonstrated that animating static pictures of objects is dependent both on the functionally relevant spatial arrangement that objects have with one another (e.g., a bottle above a glass vs. a glass above a bottle) and on the linguistic judgment to be made about those objects (e.g., "Is the bottle above the glass?" vs. "Is the bottle bigger than the glass?"). Furthermore, we showed that mental animation is driven by functional relations and language separately in the right hemisphere of the brain but conjointly in the left hemisphere. Mental animation is not a unitary construct; the predictions humans make about the visual world are driven flexibly, with hemispheric asymmetry in the routes to MT+ activation.

Curved apparent motion induced by amodal completion

We investigated whether amodal completion can bias apparent motion (AM) to deviate from its default straight path toward a longer curved path, which would violate the well-established principle that AM follows the shortest possible path. Observers viewed motion sequences of two alternating rectangular tokens positioned at the ends of a semicircular occluder, with varying interstimulus intervals (ISIs; 100-500 ms). At short ISIs, observers tended to report simple straight-path motion-that is, outside the occluder. But at long ISIs, they became increasingly likely to report a curved-path motion behind the occluder. This tendency toward reporting curved-path motion was influenced by the shape of tokens, display orientation, the gap between tokens and the occluder, and binocular depth cues. Our results suggest that the visual system tends to minimize unexplained absence of a moving object, as well as its path length, such that AM deviates from the shortest path when amodal integration of motion trajectory behind the curved occluder can account for the objective invisibility of the object during the ISI.

Investigating the anticipatory nature of pattern perception in sport

The aim of the present study was to examine the anticipatory nature of pattern perception in sport by using static and moving basketball patterns across three different display types. Participants of differing skill levels were included in order to determine whether the effects would be moderated by the knowledge and experience of the observer in the same manner reported previously for simple images. The results from a pattern recognition task showed that both expert and recreational participants were more likely to anticipate the next likely state of a pattern when it was presented as a moving video, but only the experts appeared to have the depth of understanding required to elicit the same anticipatory encoding for patterns presented as schematic images. The results extend those reported in previous research and provide further evidence of an anticipatory encoding in pattern perception for images containing complex, interrelated patterns.

Auditory temporal cues can modulate visual representational momentum

In representational momentum (RM), the final position of a moving target is mislocalized in the direction of motion. Here, the effect of a concurrent sound on visual RM was demonstrated. A visual stimulus moved horizontally and disappeared at unpredictable positions. A complex tone without any motion cues was presented continuously from the beginning of the visual motion. As compared with a silent condition, the RM magnitude increased when the sound lasted longer than and decreased when it did not last as long as the visual motion. However, the RM was unchanged when a brief complex tone was presented before or after the target disappeared (Experiment 2) or when the onset of the long-lasting sound was not synchronized with that of the visual motion (Experiments 3 and 4). These findings suggest that visual motion representation can be modulated by a sound if the visual motion information is firmly associated with the auditory information.

Cross-modal influences on representational momentum and representational gravity

Effects of cross-modal information on representational momentum and on representational gravity (ie on displacement of remembered location in the direction of target motion or in the direction of gravitational attraction, respectively) were examined. In experiment 1, ascending or descending visual motion (in the picture plane) was paired with ascending or descending auditory motion (in frequency space); motion was congruent (both ascending, both descending) or incongruent (one ascending, one descending). Memory for visual location or auditory pitch was probed. Congruence resulted in larger forward displacement for auditory pitch, but did not influence forward displacement for visual location. In experiment 2, horizontal visual motion was paired with ascending, descending, or no auditory motion. Memory for visual location was displaced downward with descending or no auditory motion, and downward displacement was larger for visual motion paired with descending auditory motion than for visual motion paired with ascending auditory motion. Effects of cross-modal information on displacement suggest representational momentum and representational gravity reflect high-level processing.

Involuntary interpretation of social cues is compromised in autism spectrum disorders

A new social distance judgment task was used to measure quantitatively the extent to which social cues are immediately and involuntary interpreted by typically developing (TD) individuals and by individuals with autism spectrum disorders (ASD). The task thus tapped into the ability to involuntary "pick up" the meaning of social cues. The cues tested were social attention and implied biological motion. Task performance of the ASD and TD groups was similarly affected by a perceptual low-level illusion induced by physical characteristics of the stimuli. In contrast, a high-level illusion induced by the implications of the social cues affected only the TD individuals; the ASD individuals remained unaffected (causing them to perform superior to TD controls). The results indicate that despite intact perceptual processing, the immediate involuntary interpretation of social cues can be compromised. We propose that this type of social cue understanding is a distinct process that should be differentiated from reflective social cue understanding and is specifically compromised in ASD. We discuss evidence for an underpinning neural substrate.

Simulation, situated conceptualization, and prediction

Based on accumulating evidence, simulation appears to be a basic computational mechanism in the brain that supports a broad spectrum of processes from perception to social cognition. Further evidence suggests that simulation is typically situated, with the situated character of experience in the environment being reflected in the situated character of the representations that underlie simulation. A basic architecture is sketched of how the brain implements situated simulation. Within this framework, simulators implement the concepts that underlie knowledge, and situated conceptualizations capture patterns of multi-modal simulation associated with frequently experienced situations. A pattern completion inference mechanism uses current perception to activate situated conceptualizations that produce predictions via simulations on relevant modalities. Empirical findings from perception, action, working memory, conceptual processing, language and social cognition illustrate how this framework produces the extensive prediction that characterizes natural intelligence.

Typical object velocity influences motion extrapolation

Previous work indicates that extrapolation of object motion during occlusion is affected by the velocity of the immediately preceding trial. Here we ask whether longer-term velocity representations can also influence motion extrapolation. Red, blue or green targets disappeared behind an occluder. Participants pressed a button when they thought the target had reached the other side. Red targets were slower (10-20 deg/s), blue targets moved at medium velocities (14-26 deg/s) and green targets were faster (20-30 deg/s). We compared responses on a subset of red and green trials which always travelled at 20 deg/s. Although trial velocities were identical, participants responded as if the green targets moved faster (M = 22.64 deg/s) then the red targets (M = 19.72 deg/s). This indicates that motion extrapolation is affected by longer-term information about the typical velocity of different categories of stimuli.

Integrated contextual representation for objects' identities and their locations

Visual context plays a prominent role in everyday perception. Contextual information can facilitate recognition of objects within scenes by providing predictions about objects that are most likely to appear in a specific setting, along with the locations that are most likely to contain objects in the scene. Is such identity-related ("semantic") and location-related ("spatial") contextual knowledge represented separately or jointly as a bound representation? We conducted a functional magnetic resonance imaging (fMRI) priming experiment whereby semantic and spatial contextual relations between prime and target object pictures were independently manipulated. This method allowed us to determine whether the two contextual factors affect object recognition with or without interacting, supporting a unified versus independent representations, respectively. Results revealed a Semantic x Spatial interaction in reaction times for target object recognition. Namely, significant semantic priming was obtained when targets were positioned in expected (congruent), but not in unexpected (incongruent), locations. fMRI results showed corresponding interactive effects in brain regions associated with semantic processing (inferior prefrontal cortex), visual contextual processing (parahippocampal cortex), and object-related processing (lateral occipital complex). In addition, activation in fronto-parietal areas suggests that attention and memory-related processes might also contribute to the contextual effects observed. These findings indicate that object recognition benefits from associative representations that integrate information about objects' identities and their locations, and directly modulate activation in object-processing cortical regions. Such context frames are useful in maintaining a coherent and meaningful representation of the visual world, and in providing a platform from which predictions can be generated to facilitate perception and action.

Spatial memory and explicit knowledge: an effect of instruction on representational momentum

Freyd (1987; Finke & Freyd, 1985) suggested that representational momentum (i.e., forward displacement in memory for the location of a moving target) is impervious to error feedback (i.e., is modular or cognitively impenetrable), but studies supporting this claim might not have allowed sufficient opportunity for learning to occur. In the experiment reported here, participants were (a) naive regarding representational momentum, (b) informed about representational momentum but not instructed to counteract it, or (c) informed about representational momentum and instructed to counteract it. All participants exhibited significant displacement. However, participants informed about representational momentum exhibited less forward displacement than did naive participants due to a greater tendency to respond same to probes behind the true--same position. Possible mechanisms of compensation and the notion that displacement reflects both modular (cognitively impenetrable) and nonmodular (cognitively penetrable) components are addressed.

Grounded cognition

Grounded cognition rejects traditional views that cognition is computation on amodal symbols in a modular system, independent of the brain's modal systems for perception, action, and introspection. Instead, grounded cognition proposes that modal simulations, bodily states, and situated action underlie cognition. Accumulating behavioral and neural evidence supporting this view is reviewed from research on perception, memory, knowledge, language, thought, social cognition, and development. Theories of grounded cognition are also reviewed, as are origins of the area and common misperceptions of it. Theoretical, empirical, and methodological issues are raised whose future treatment is likely to affect the growth and impact of grounded cognition.

Prospective coding in event representation

A perceived event such as a visual stimulus in the external world and a to-be-produced event such as an intentional action are subserved by event representations. Event representations do not only contain information about present states but also about past and future states. Here we focus on the role of representing future states in event perception and generation (i.e., prospective coding). Relevant theoretical issues and paradigms are discussed. We suggest that the predictive power of the motor system may be exploited for prospective coding not only in producing but also in perceiving events. Predicting is more advantageous than simply reacting. Perceptual prediction allows us to select appropriate responses ahead of the realization of an (anticipated) event and therefore, it is indispensable to flexibly and timely adapt to new situations and thus, successfully interact with our physical and social environment.

Naive beliefs in baseball: systematic distortion in perceived time of apex for fly balls

When fielders catch fly balls they use geometric properties to optically maintain control over the ball. The strategy provides ongoing guidance without indicating precise positional information concerning where the ball is located in space. Here, the authors show that observers have striking misconceptions about what the motion of projectiles should look like from various perspectives and that they estimate when the physical apex of a fly ball occurs to be far later than actual, irrespective of baseball experience. Their estimations are consistent with the highest point they are looking at as the ball approaches, not with the physical apex. These findings introduce a new and robust effect in intuitive perception in which people confuse their perceptual perspective with the physical situation that they mentally represent.

Comment and Reply Why eye movements and perceptual factors have to be controlled in studies on “representational momentum”

In order to study memory of the final position of a smoothly moving target, Hubbard (e.g., Hubbard and Bharucha, 1988) presented smooth stimulus motion and used motor responses. In contrast, Freyd (e.g., Freyd and Finke, 1984) presented implied stimulus motion and used the method of constant stimuli. The same forward error was observed in both paradigms. However, the processes underlying the error may be very different. When smooth stimulus motion is followed by smooth pursuit eye movements, the forward error is associated with asynchronous processing of retinal and extraretinal information. In the absence of eye movements, no forward displacement is observed with smooth motion. In contrast, implied motion produces a forward error even without eye movements, suggesting that observers extrapolate the next target step when successive target presentations are far apart. Finally, motor responses produce errors that are not observed with perceptual judgments, indicating that the motor system may compensate for neuronal latencies.

Higher order pattern structure influences auditory representational momentum

Representational momentum refers to the phenomenon that observers tend to incorrectly remember an event undergoing real or implied motion as shifted beyond its actual final position. This has been demonstrated in both visual and auditory domains. In 5 pitch discrimination experiments, listeners heard tone sequences that implied either linear, periodic, or null motions in pitch space. Their task was to judge whether the pitch of a probe tone following each sequence was the same or different from the final sequence tone. Results suggested that listeners made errors consistent with extrapolation of coherent pitch patterns (linear, periodic) but not with incoherent (null) ones. Hypotheses associated with internalized physical principles and pattern-based expectations are discussed.

Representational momentum and related displacements in spatial memory: A review of the findings

Memory for the final location of a moving target is often displaced in the direction of target motion, and this has been referred to as representational momentum. Characteristics of the target (e.g., velocity, size, direction, and identity), display (e.g., target format, retention interval, and response method), context (landmarks, expectations, and attribution of motion source), and observer (e.g., allocation of attention, eye movements, and psychopathology) that influence the direction and magnitude of displacement are reviewed. Specific conclusions regarding numerous variables that influence displacement (e.g., presence of landmarks or surrounding context), as well as broad-based conclusions regarding displacement in general (e.g., displacement does not reflect objective physical principles, may reflect aspects of naive physics, does not solely reflect eye movements, may involve some modular processing, and reflects high-level processes) are drawn. A possible computational theory of displacement is suggested in which displacement (1) helps bridge the gap between perception and action and (2) plays a critical part in localizing stimuli in the environment.

Illusory motion and representational momentum

When a moving target vanishes abruptly, participants judge its final position as being ahead of its actual final position, in the direction of motion (representational momentum; Freyd & Finke, 1984). In the present study, we presented illusory motion and examined whether or not forward displacement was affected by the perceived direction and speed of the target. Experiments 1A and 1B showed that an illusory direction of movement of a target was perceived, and Experiment 2 showed that an illusory speed of a moving target was observed. However, neither the direction nor the magnitude of forward displacement was affected by these illusions. Therefore, it was suggested that the mechanism underlying forward displacement (or some extrapolation processing) uses different motion signals than does the perceptual mechanism.

The Case for Motor Involvement in Perceiving Conspecifics

Perceiving other people's behaviors activates imitative motor plans in the perceiver, but there is disagreement as to the function of this activation. In contrast to other recent proposals (e.g., that it subserves overt imitation, identification and understanding of actions, or working memory), here it is argued that imitative motor activation feeds back into the perceptual processing of conspecifics' behaviors, generating top-down expectations and predictions of the unfolding action. Furthermore, this account incorporates recent ideas about emulators in the brain-mental simulations that run in parallel to the external events they simulate-to provide a mechanism by which motoric involvement could contribute to perception. Evidence from a variety of literatures is brought to bear to support this account of perceiving human body movement.

Spatial perception and control

We investigated whether the perceived vanishing point of a moving stimulus becomes more accurate as one's degree of control over the stimulus increases. Either alone or as a member of a pair, participants controlled the progression of a dot stimulus back and forth across a computer monitor. They did so via right and left buttonpresses that incremented the dot's velocity rightward and leftward, respectively. The participants in the individual condition had control of both buttons. Those in the group condition had control of only one. As the participants slowed the dot to change its direction of travel, it unexpectedly disappeared. Localizations of the vanishing point became more accurate as the participants' control over the dot increased. The data bridge a gap between accounts of localization error that rely solely on stimulus and cognitive factors, and accounts derived from research on action and spatial perception, which tend to rely on action-planning factors.

Attention maintains mental extrapolation of target position: irrelevant distractors eliminate forward displacement after implied motion

Observers' judgments of the final position of a moving target are typically shifted in the direction of implied motion ("representational momentum"). The role of attention is unclear: visual attention may be necessary to maintain or halt target displacement. When attention was captured by irrelevant distractors presented during the retention interval, forward displacement after implied target motion disappeared, suggesting that attention may be necessary to maintain mental extrapolation of target motion. In a further corroborative experiment, the deployment of attention was measured after a sequence of implied motion, and faster responses were observed to stimuli appearing in the direction of motion. Thus, attention may guide the mental extrapolation of target motion. Additionally, eye movements were measured during stimulus presentation and retention interval. The results showed that forward displacement with implied motion does not depend on eye movements. Differences between implied and smooth motion are discussed with respect to recent neurophysiological findings.

“Representational Momentum“:

Zusammenfassung. Die Hypothesen, dass mentale Repräsentationen inhärent dynamisch seien, und dass diese Dynamik den physikalischen Gesetzmäßigkeiten der realen Welt folge, werden diskutiert. Als Belege für diese Hypothesen galten Fehlleistungen des visuellen Kurzzeitgedächtnisses: Die erinnerte letzte Position eines bewegten Zielreizes wich horizontal in Bewegungsrichtung und vertikal nach unten von der tatsächlichen ab, so als ob die Repräsentation des Objektes Impuls (“representational momentum“) und Schwerkraft besäße. Weitere Gedächtnisverschiebungen ließen darauf schließen, dass es auch mentale Analoga von Gewicht, Reibung und Zentripetalkraft gibt. Die empirische Evaluation der Befunde wirft allerdings Zweifel an der Validität der experimentellen Manipulationen und Schlussfolgerungen auf. Es wurde entweder ein sehr eigentümliches experimentelles Design verwendet, oder die Blickposition wurde nicht kontrolliert. Beide Faktoren verändern die Ergebnisse maßgeblich. Neuere Experimente weisen darauf hin, dass Täuschungen im Wahrnehmungsprozess, nicht aber die Dynamik der mentalen Repräsentationen, die Gedächtnisverzerrungen produziert. Blickbewegungen sind für einen Teil der perzeptuellen Fehlleistungen verantwortlich.

The locus of "memory displacement" is at least partially perceptual: effects of velocity, expectation, friction, memory averaging, and weight

When observers are asked to localize the final position of a moving target, the judged position is usually displaced from the actual position. It has been suggested that mental processes derived from a number of invariant and noninvariant principles produce the mislocalization in memory. In this study, the effects of velocity, expectation, friction, memory averaging, and weight were reconsidered, and evidence was accumulated that supports the alternative view that the distortions arise to a large degree at a perceptual level. Effects of velocity and expectation were present when observers still perceived a persisting image of the target. It is suggested that the active reorienting of the perceptual organs explains the distortions. Furthermore, distortions of the perceived center of a visible stimulus may explain effects that have previously been attributed to memory averaging and mental analogues of weight. Thus, the locus of memory displacement is at least partially perceptual.

Implicit motion perception in schizotypy and schizophrenia: a Representational Momentum study

INTRODUCTION

Human observers exhibit a distortion in recognition memory for pictures that imply motion because of an automatic mental process, which extrapolates along the implied trajectory of the picture. This is known as Representational Momentum (RM). Converging evidence (functional imaging; magnetic stimulation studies) suggests activity in area MT/MST (V5) is necessary for RM to occur. Patients with schizophrenia and healthy schizotypic individuals have been found to show motion perception deficits and abnormal eye-tracking (both indicative of abnormal functioning within brain area V5), therefore it was hypothesised that these individuals would show a reduced or absent RM effect.

METHOD

Fifty healthy individuals and seven patients diagnosed with schizophrenia undertook a task previously found to elicit the RM effect.

RESULTS

Although the size of the RM effect was not significantly different between either low and high schizotypes or low schizotypes and patients, there was a trend (in the opposite direction to that predicted) for the patients with schizophrenia and the high schizotypes to exhibit a larger RM effect.

CONCLUSION

The findings are discussed in terms of functional connectivity between frontal areas and V5, and of schizophrenia involving a failure to inhibit automatic processes.