A visual illusion that influences perception and action through the dorsal pathway

Entrainment of visuomotor responses to target speed during interception

Motor actions adapt dynamically to external changes through the brain's ability to predict sensory outcomes and adjust for discrepancies between anticipated and actual sensory inputs. In this study, we investigated how changes in target speed (vT) and direction influenced visuomotor responses, focusing on gaze and manual joystick control during an interception task. Participants tracked a moving target with sinusoidal variations in vT and directional changes, generating sensory prediction errors and requiring real-time adjustments. Our results demonstrate slow variations in vT entrained gaze and joystick metrics, with participants synchronizing their responses to the cycles of target motion. While target directional changes alone had limited impact, combining them with sinusoidal variations in vT led to robust behavioral entrainment. Participants also exhibited rapid within-trial adjustments, with peak gaze and joystick gains increasing linearly with vT frequency, highlighting the critical role of manual control in matching or exceeding vT for successful interception. Additionally, responses to sudden phase changes in the vT sinusoid revealed the continuous monitoring of prediction errors driven by the magnitude of phase shifts. These findings illustrate the brain's predictive system's ability to integrate continuous visual feedback and sensory prediction errors to fine-tune motor responses and anticipate future target speeds.

A continuum from predictive to online feedback in visuomotor interception

Interception, essential for activities like driving and sports, can be characterized by varying degrees of predictive behaviour. We developed a visually guided task to explore how target predictability and visibility influenced interception actions. The task featured a falling dot influenced by horizontal velocity, gravity and air friction, with predictability manipulated through external forces that altered the target's trajectory. We also introduced spatial occlusion to limit visual information. Our results show that low target variability favoured predictive behaviours, while high variability led to more reactive responses relying on online feedback. Manual responses displayed increased variability with changes in target motion, whereas eye trajectories maintained constant curvature across conditions. Additionally, higher target variability delayed the onset of hand movements but did not affect eye movement onset, making gaze position a poor predictor of hand position. This distinction highlights the different adaptive patterns in hand and eye movements in response to target trajectory changes. Participants maintained stable interception behaviours within and across sessions, indicating individual preferences for either predictive or more reactive actions. Our findings reveal a dynamic interplay between target predictability and interception, illustrating how humans combine predictive and reactive behaviours to manage external variability.

The "What" and "How" of Pantomime Actions

Pantomimes are human actions that simulate ideas, objects, and events, commonly used in conversation, performance art, and gesture-based interfaces for computing and controlling robots. Yet, their underlying neurocognitive mechanisms are not well understood. In this review, we examine pantomimes through two parallel lines of research: (1) the two visual systems (TVS) framework for visually guided action, and (2) the neuropsychological literature on limb apraxia. Historically, the TVS framework has considered pantomime actions as expressions of conscious perceptual processing in the ventral stream, but an emerging view is that they are jointly influenced by ventral and dorsal stream processing. Within the apraxia literature, pantomimes were historically viewed as learned motor schemas, but there is growing recognition that they include creative and improvised actions. Both literatures now recognize that pantomimes are often created spontaneously, sometimes drawing on memory and always requiring online cognitive control. By highlighting this convergence of ideas, we aim to encourage greater collaboration across these two research areas, in an effort to better understand these uniquely human behaviors.

Virtual reality perceptual training can improve the temporal discrimination ability of swinging during softball batting

Perception and action uncoupling in fastball sports anticipatory training is often criticized. Nevertheless, perception-only training offers distinct advantages, such as flexibility concerning time, place, and injury limitations. Therefore, the effectiveness of this training approach warrants evaluation. This study developed a virtual reality (VR) training system based on the idea that the two visual pathways in the brain are associated with visual information attributes, rather than perception or action. A key feature of this study's perception-only training was its presentation of not only the opponent's kinematics but also the ball's flight information (the attributes that guide hitting) to train the visual system necessary for real situations. Seventeen female softball batters were assigned to two groups: a training group (N = 9) and a control group (N = 8). Only the training group underwent the VR anticipatory skill training to discriminate the different ball speeds. Both groups completed a perception test and an action test on the VR during the pre- and post-training periods. The perception test assessed response accuracy (RA) in discriminating ball speed, and the action test assessed the temporal difference in swing onset (delta onset). Comparison of these two outcome variables between groups revealed that perception-only training improved both perceptual and action responses. This study demonstrated the effectiveness of perception-only training and emphasized the need for its thoughtful utilization and design.

Spatial contextual cues that help predict how a target will accelerate can be used to guide interception

Objects in one's environment do not always move at a constant velocity but often accelerate or decelerate. People are very poor at visually judging acceleration and normally make systematic errors when trying to intercept accelerating objects. If the acceleration is perpendicular to the direction of motion, it gives rise to a curved path. Can spatial contextual cues help one predict such accelerations and thereby help interception? To answer this question, we asked participants to hit a target that moved as if it were attached to a rolling disk, like a valve (target) on a bicycle wheel (disk) moves when cycling: constantly accelerating toward the wheel's center. On half the trials, the disk was visible such that participants could use the spatial relations between the target and the rolling disk to guide their interception. On the other half, the disk was not visible, so participants had no help in predicting the target's complicated pattern of accelerations and decelerations. Importantly, the target's path was the same in both cases. Participants hit more targets when the disk was visible than when it was invisible, even when using a strategy that can compensate for neglecting acceleration. We conclude that spatial contextual cues that help predict the target's accelerations can help intercept it.

Neural networks underlying visual illusions: An activation likelihood estimation meta-analysis

Visual illusions have long been used to study visual perception and contextual integration. Neuroimaging studies employ illusions to identify the brain regions involved in visual perception and how they interact. We conducted an Activation Likelihood Estimation (ALE) meta-analysis and meta-analytic connectivity modeling on fMRI studies using static and motion illusions to reveal the neural signatures of illusory processing and to investigate the degree to which different areas are commonly recruited in perceptual inference. The resulting networks encompass ventral and dorsal regions, including the inferior and middle occipital cortices bilaterally in both types of illusions. The static and motion illusion networks selectively included the right posterior parietal cortex and the ventral premotor cortex respectively. Overall, these results describe a network of areas crucially involved in perceptual inference relying on feed-back and feed-forward interactions between areas of the ventral and dorsal visual pathways. The same network is proposed to be involved in hallucinogenic symptoms characteristic of schizophrenia and other disorders, with crucial implications in the use of illusions as biomarkers.

Category-specific and category-general neural codes of recognition memory in the ventral visual pathway

Researchers have identified category-specific brain regions, such as the fusiform face area (FFA) and parahippocampal place area (PPA) in the ventral visual pathway, which respond preferentially to one particular category of visual objects. In addition to their category-specific role in visual object identification and categorization, regions in the ventral visual pathway play critical roles in recognition memory. Nevertheless, it is not clear whether the contributions of those brain regions to recognition memory are category-specific or category-general. To address this question, the present study adopted a subsequent memory paradigm and multivariate pattern analysis (MVPA) to explore category-specific and category-general neural codes of recognition memory in the visual pathway. The results revealed that the right FFA and the bilateral PPA showed category-specific neural patterns supporting recognition memory of faces and scenes, respectively. In contrast, the lateral occipital cortex seemed to carry category-general neural codes of recognition memory. These results provide neuroimaging evidence for category-specific and category-general neural mechanisms of recognition memory in the ventral visual pathway.

The cost of aiming for the best answers: Inconsistent perception

The laws of physics and mathematics describe the world we live in as internally consistent. As these rules provide a very effective description, and our interaction with the world is also very effective, it seems self-evident that our perception follows these laws. As a result, when trying to explain imperfections in perception, we tend to impose consistency and introduce concepts such as deformations of visual space. In this review, we provide numerous examples that show that in many situations we perceive related attributes to have inconsistent values. We discuss how our tendency to assume consistency leads to erroneous conclusions on how we process sensory information. We propose that perception is not about creating a consistent internal representation of the outside world, but about answering specific questions about the outside world. As the information used to answer a question is specific for that question, this naturally leads to inconsistencies in perception and to an apparent dissociation between some perceptual judgments and related actions.

Optimal integration of kinematic and ball-flight information when perceiving the speed of a moving ball

In order to intercept a moving target such as a baseball with high spatio-temporal accuracy, the perception of the target's movement speed is important for estimating when and where the target will arrive. However, it is unclear what sources of information are used by a batter to estimate ball speed and how those sources of information are integrated to facilitate successful interception. In this study, we examined the degree to which kinematic and ball-flight information are integrated when estimating ball speed in baseball batting. Thirteen university level baseball batters performed a ball-speed evaluation task in a virtual environment where they were required to determine which of two comparison baseball pitches (i.e., a reference and comparison stimuli) they perceived to be faster. The reference and comparison stimuli had the same physical ball speed, but with different pitching movement speeds in the comparison stimuli. The task was performed under slow (125 km/h) and fast (145 km/h) ball-speed conditions. Results revealed that the perceived ball-speed was influenced by the movement speed of the pitcher's motion, with the influence of the pitcher's motion more pronounced in the fast ball-speed condition when ball-flight information was presumably less reliable. Moreover, exploratory analyses suggested that the more skilled batters were increasingly likely to integrate the two sources of information according to their relative reliability when making judgements of ball speed. The results provide important insights into how skilled performers may make judgements of speed and time to contact, and further enhance our understanding of how the ability to make those judgements might improve when developing expertise in hitting.

Spatial distances affect temporal prediction and interception

The more distant two consecutive stimuli are presented, the longer the temporal interstimulus interval (ISI) between their presentations is perceived (kappa effect). The present study aimed at testing whether the kappa effect not only affects perceptual estimates of time, but also motor action, more specifically, interception. In a first step, the original kappa paradigm was adapted to assess the effect in temporal prediction. Second, the task was further modified to an interception task, requiring participants to spatially and temporally predict and act. In two online experiments, a white circle was successively presented at three locations moving from left to right with constant spatial and temporal ISIs in between. Participants were asked to either (i) indicate the time of appearance of the predicted fourth stimulus (Exp. 1) or to (ii) intercept the predicted fourth location at the correct time (Exp. 2). In both experiments the temporal response depended on the spatial intervals. In line with the kappa effect, participants predicted the final stimulus to appear later (Exp. 1) or intercepted it later (Exp. 2), the more distant the stimuli were presented. Together, these results suggest that perceptual biases such as the kappa effect impact motor interception performance.

Tau and kappa in interception - how perceptual spatiotemporal interrelations affect movements

Batting and catching are real-life examples of interception. Due to latencies between the processing of sensory input and the corresponding motor response, successful interception requires accurate spatiotemporal prediction. However, spatiotemporal predictions can be subject to bias. For instance, the more spatially distant two sequentially presented objects are, the longer the interval between their presentations is perceived (kappa effect) and vice versa (tau effect). In this study, we deployed these phenomena to test in two sensory modalities whether temporal representations depend asymmetrically on spatial representations, or whether both are symmetrically interrelated. We adapted the tau and kappa paradigms to an interception task by presenting four stimuli (visually or auditorily) one after another on four locations, from left to right, with constant spatial and temporal intervals in between. In two experiments, participants were asked to touch the screen where and when they predicted a fifth stimulus to appear. In Exp. 2, additional predictive gaze measures were examined. Across experiments, auditory but not visual stimuli produced a tau effect for interception, supporting the idea that the relationship between space and time is moderated by the sensory modality. Results did not reveal classical auditory or visual kappa effects and no visual tau effects. Gaze data in Exp. 2 showed that the (spatial) gaze orientation depended on temporal intervals while the timing of fixations was modulated by spatial intervals, thereby indicating tau and kappa effects across modalities. Together, the results suggest that sensory modality plays an important role in spatiotemporal predictions in interception.

Effect of Manipulating Advanced Kinematic Information on Hitting Movement Prediction, Perception, and Action

Purpose: This study examined the effect of manipulating advanced kinematic information about opponents' pitching movement on ball speed prediction, ball speed perception, and impact timing errors under strict temporal constraints (i.e., a softball game). Method: Three experiments were conducted using visual stimuli consisting of varied kinematic information-different pitching movements with the same ball trajectory. In Experiment 1, participants observed the pitching movement of the visual stimuli and predicted pitched ball speed as a two-interval forced choice discrimination task (2IFC). In Experiment 2, participants observed both the pitching movement and ball trajectory, and evaluated the pitched ball speed as a 2IFC. In Experiments 3a and 3b, they tried to swing against the pitched ball presented on the screen as accurately as possible with regard to timing. Results: Batters tended to predict the ball was moving faster when the pitching movement was faster (Experiment 1). Incorrectly predicting the ball speed due to the difference in advanced kinematic information also biased batters' perception of the speed (Experiment 2), and this biased prediction yielded congruent impact timing (Experiment 3a). The impact timing error of naive participants also was affected by kinematic information (Experiment 3b). Conclusion: Limitations of this study (representative task design, sample size, and experimental procedures) notwithstanding, results indicate that, under strict temporal constraints, batters' perceptions and actions are sensitive to advanced kinematic information, which could lead to biased perceptions and actions.

Motion-in-depth effects on interceptive timing errors in an immersive environment

We often need to interact with targets that move along arbitrary trajectories in the 3D scene. In these situations, information of parameters like speed, time-to-contact, or motion direction is required to solve a broad class of timing tasks (e.g., shooting, or interception). There is a large body of literature addressing how we estimate different parameters when objects move both in the fronto-parallel plane and in depth. However, we do not know to which extent the timing of interceptive actions is affected when motion-in-depth (MID) is involved. Unlike previous studies that have looked at the timing of interceptive actions using constant distances and fronto-parallel motion, we here use immersive virtual reality to look at how differences in the above-mentioned variables influence timing errors in a shooting task performed in a 3D environment. Participants had to shoot at targets that moved following different angles of approach with respect to the observer when those reached designated shooting locations. We recorded the shooting time, the temporal and spatial errors and the head's position and orientation in two conditions that differed in the interval between the shot and the interception of the target's path. Results show a consistent change in the temporal error across approaching angles: the larger the angle, the earlier the error. Interestingly, we also found different error patterns within a given angle that depended on whether participants tracked the whole target's trajectory or only its end-point. These differences had larger impact when the target moved in depth and are consistent with underestimating motion-in-depth in the periphery. We conclude that the strategy participants use to track the target's trajectory interacts with MID and affects timing performance.

Overt and Covert Object Features Mediate Timing of Patterned Brain Activity during Motor Planning

Humans are seamless in their ability to efficiently and reliably generate fingertip forces to gracefully interact with objects. Such interactions rarely end in awkward outcomes like spilling, crushing, or tilting given advanced motor planning. Here we combine multiband imaging with deconvolution- and Bayesian pattern component modeling of functional magnetic resonance imaging data and in-scanner kinematics, revealing compelling evidence that the human brain differentially represents preparatory information for skillful object interactions depending on the saliency of visual cues. Earlier patterned activity was particularly evident in ventral visual processing stream-, but also selectively in dorsal visual processing stream and cerebellum in conditions of heightened uncertainty when an object’s superficial shape was incompatible rather than compatible with a key underlying object feature.

Perceptual decisions about object shape bias visuomotor coordination during rapid interception movements

Visual processing for perception and for action is thought to be mediated by two specialized neural pathways. Using a visuomotor decision-making task, we show that participants differentially utilized online perceptual decision-making in reaching and interception and that eye movements necessary for perception influenced motor decision strategies. These results provide evidence that task complexity modulates how pathways processing perception versus action information interact during the visual control of movement.

Spatial coding for memory-guided reaching in visual and pictorial spaces

An essential difference between pictorial space displayed as paintings, photographs, or computer screens, and the visual space experienced in the real world is that the observer has a defined location, and thus valid information about distance and direction of objects, in the latter but not in the former. Thus egocentric information should be more reliable in visual space, whereas allocentric information should be more reliable in pictorial space. The majority of studies relied on pictorial representations (images on a computer screen), leaving it unclear whether the same coding mechanisms apply in visual space. Using a memory-guided reaching task in virtual reality, we investigated allocentric coding in both visual space (on a table in virtual reality) and pictorial space (on a monitor that is on the table in virtual reality). Our results suggest that the brain uses allocentric information to represent objects in both pictorial and visual space. Contrary to our hypothesis, the influence of allocentric cues was stronger in visual space than in pictorial space, also after controlling for retinal stimulus size, confounding allocentric cues, and differences in presentation depth. We discuss possible reasons for stronger allocentric coding in visual than in pictorial space.

Spatial updating of allocentric landmark information in real-time and memory-guided reaching

The 2-streams model of vision suggests that egocentric and allocentric reference frames are utilized by the dorsal and the ventral stream for real-time and memory-guided movements, respectively. Recent studies argue against such a strict functional distinction and suggest that real-time and memory-guided movements recruit the same spatial maps. In this study we focus on allocentric spatial coding and updating of targets by using landmark information in real-time and memory-guided reaching. We presented participants with a naturalistic scene which consisted of six objects on a table that served as potential reach targets. Participants were informed about the target object after scene encoding, and were prompted by a go cue to reach to its position. After target identification a brief air-puff was applied to the participant's right eye inducing an eye blink. During the blink the target object disappeared from the scene, and in half of the trials the remaining objects, that functioned as landmarks, were shifted horizontally in the same direction. We found that landmark shifts systematically influenced participants' reaching endpoints irrespective of whether the movements were controlled online based on available target information (real-time movement) or memory-guided based on remembered target information (memory-guided movement). Overall, the effect of landmark shift was stronger for memory-guided than real-time reaching. Our findings suggest that humans can encode and update reach targets in an allocentric reference frame for both real-time and memory-guided movements and show stronger allocentric coding when the movement is based on memory.

The predictability of a target’s motion influences gaze, head, and hand movements when trying to intercept it

Does the predictability of a target’s movement and of the interception location influence how the target is intercepted? In a first experiment, we manipulated the predictability of the interception location. A target moved along a haphazardly curved path, and subjects attempted to tap on it when it entered a hitting zone. The hitting zone was either a large ring surrounding the target’s starting position (ring condition) or a small disk that became visible before the target appeared (disk condition). The interception location gradually became apparent in the ring condition, whereas it was immediately apparent in the disk condition. In the ring condition, subjects pursued the target with their gaze. Their heads and hands gradually moved in the direction of the future tap position. In the disk condition, subjects immediately directed their gaze toward the hitting zone by moving both their eyes and heads. They also moved their hands to the future tap position sooner than in the ring condition. In a second and third experiment, we made the target’s movement more predictable. Although this made the targets easier to pursue, subjects now shifted their gaze to the hitting zone soon after the target appeared in the ring condition. In the disk condition, they still usually shifted their gaze to the hitting zone at the beginning of the trial. Together, the experiments show that predictability of the interception location is more important than predictability of target movement in determining how we move to intercept targets.

NEW & NOTEWORTHY We show that if people are required to intercept a target at a known location, they direct their gaze to the interception point as soon as they can rather than pursuing the target with their eyes for as long as possible. The predictability of the interception location rather than the predictability of the path to that location largely determines how the eyes, head, and hand move.