Differences in the Accuracy of Human Visuospatial Memory After Yaw and Roll Rotations

Psychophysical evidence for the involvement of head/body-centered reference frames in egocentric visuospatial memory: A whole-body roll tilt paradigm

Accurate memory regarding the location of an object with respect to one's own body, termed egocentric visuospatial memory, is essential for action directed toward the object. Although researchers have suggested that the brain stores information related to egocentric visuospatial memory not only in the eye-centered reference frame but also in the other egocentric (i.e., head- or body-centered or both) reference frames, experimental evidence is scarce. Here, we tested this possibility by exploiting the perceptual distortion of head/body-centered coordinates via whole-body tilt relative to gravity. We hypothesized that if the head/body-centered reference frames are involved in storing the egocentric representation of a target in memory, then reproduction would be affected by this perceptual distortion. In two experiments, we asked participants to reproduce the remembered location of a visual target relative to their head/body. Using intervening whole-body roll rotations, we manipulated the initial (target presentation) and final (reproduction of the remembered location) body orientations in space and evaluated the effect on the reproduced location. Our results showed significant biases of the reproduced target location and perceived head/body longitudinal axis in the direction of the intervening body rotation. Importantly, the amount of error was correlated across participants. These results provide experimental evidence for the neural encoding and storage of information related to egocentric visuospatial memory in the head/body-centered reference frames.

The influence of yaw rotation on spatial navigation during development

Sensory cues enable navigation through space, as they inform us about movement properties, such as the amount of travelled distance and the heading direction. In this study, we focused on the ability to spatially update one's position when only proprioceptive and vestibular information is available. We aimed to investigate the effect of yaw rotation on path integration across development in the absence of visual feedback. To this end, we utilized the triangle completion task: participants were guided through two legs of a triangle and asked to close the shape by walking along its third imagined leg. To test the influence of yaw rotation across development, we tested children between 6 and 11 years old (y.o.) and adults on their perceptions of angles of different degrees. Our results demonstrated that the amount of turn while executing the angle influences performance at all ages, and in some aspects, also interacted with age. Indeed, whilst adults seemed to adjust their heading towards the end of their walked path, younger children took less advantage of this strategy. The amount of disorientation the path induced also affected participants' full maturational ability to spatially navigate with no visual feedback. Increasing induced disorientation required children to be older to reach adult-level performance. Overall, these results provide novel insights on the maturation of spatial navigation-related processes.

Weighted Visual and Vestibular Cues for Spatial Updating During Passive Self-Motion

When walking or driving, it is of the utmost importance to continuously track the spatial relationship between objects in the environment and the moving body in order to prevent collisions. Although this process of spatial updating occurs naturally, it involves the processing of a myriad of noisy and ambiguous sensory signals. Here, using a psychometric approach, we investigated the integration of visual optic flow and vestibular cues in spatially updating a remembered target position during a linear displacement of the body. Participants were seated on a linear sled, immersed in a stereoscopic virtual reality environment. They had to remember the position of a target, briefly presented before a sideward translation of the body involving supra-threshold vestibular cues and whole-field optic flow that provided slightly discrepant motion information. After the motion, using a forced response participants indicated whether the location of a brief visual probe was left or right of the remembered target position. Our results show that in a spatial updating task involving passive linear self-motion humans integrate optic flow and vestibular self-displacement information according to a weighted-averaging process with, across subjects, on average about four times as much weight assigned to the visual compared to the vestibular contribution (i.e., 79% visual weight). We discuss our findings with respect to previous literature on the effect of optic flow on spatial updating performance.

Evidence for a reference frame transformation of vestibular signal contributions to voluntary reaching

To contribute appropriately to voluntary reaching during body motion, vestibular signals must be transformed from a head-centered to a body-centered reference frame. We quantitatively investigated the evidence for this transformation during online reach execution by using galvanic vestibular stimulation (GVS) to simulate rotation about a head-fixed, roughly naso-occipital axis as human subjects made planar reaching movements to a remembered location with their head in different orientations. If vestibular signals that contribute to reach execution have been transformed from a head-centered to a body-centered reference frame, the same stimulation should be interpreted as body tilt with the head upright but as vertical-axis rotation with the head inclined forward. Consequently, GVS should perturb reach trajectories in a head-orientation-dependent way. Consistent with this prediction, GVS applied during reach execution induced trajectory deviations that were significantly larger with the head forward compared with upright. Only with the head forward were trajectories consistently deviated in opposite directions for rightward versus leftward simulated rotation, as appropriate to compensate for body vertical-axis rotation. These results demonstrate that vestibular signals contributing to online reach execution have indeed been transformed from a head-centered to a body-centered reference frame. Reach deviation amplitudes were comparable to those predicted for ideal compensation for body rotation using a biomechanical limb model. Finally, by comparing the effects of application of GVS during reach execution versus prior to reach onset we also provide evidence that spatially transformed vestibular signals contribute to at least partially distinct compensation mechanisms for body motion during reach planning versus execution.

Visuo-vestibular interaction: predicting the position of a visual target during passive body rotation

Following body rotation, optimal updating of the position of a memorized target is attained when retinal error is perceived and corrective saccade is performed. Thus, it appears that these processes may enable the calibration of the vestibular system by facilitating the sharing of information between both reference frames. Here, it is assessed whether having sensory information regarding body rotation in the target reference frame could enhance an individual's learning rate to predict the position of an earth-fixed target. During rotation, participants had to respond when they felt their body midline had crossed the position of the target and received knowledge of result. During practice blocks, for two groups, visual cues were displayed in the same reference frame of the target, whereas a third group relied on vestibular information (vestibular-only group) to predict the location of the target. Participants, unaware of the role of the visual cues (visual cues group), learned to predict the location of the target and spatial error decreased from 16.2 to 2.0°, reflecting a learning rate of 34.08 trials (determined from fitting a falling exponential model). In contrast, the group aware of the role of the visual cues (explicit visual cues group) showed a faster learning rate (i.e. 2.66 trials) but similar final spatial error 2.9°. For the vestibular-only group, similar accuracy was achieved (final spatial error of 2.3°), but their learning rate was much slower (i.e. 43.29 trials). Transferring to the Post-test (no visual cues and no knowledge of result) increased the spatial error of the explicit visual cues group (9.5°), but it did not change the performance of the vestibular group (1.2°). Overall, these results imply that cognition assists the brain in processing the sensory information within the target reference frame.

Absence of spatial updating when the visuomotor system is unsure about stimulus motion

How does the visuomotor system decide whether a target is moving or stationary in space or whether it moves relative to the eyes or head? A visual flash during a rapid eye-head gaze shift produces a brief visual streak on the retina that could provide information about target motion, when appropriately combined with eye and head self-motion signals. Indeed, double-step experiments have demonstrated that the visuomotor system incorporates actively generated intervening gaze shifts in the final localization response. Also saccades to brief head-fixed flashes during passive whole-body rotation compensate for vestibular-induced ocular nystagmus. However, both the amount of retinal motion to invoke spatial updating and the default strategy in the absence of detectable retinal motion remain unclear. To study these questions, we determined the contribution of retinal motion and the vestibular canals to spatial updating of visual flashes during passive whole-body rotation. Head- and body-restrained humans made saccades toward very brief (0.5 and 4 ms) and long (100 ms) visual flashes during sinusoidal rotation around the vertical body axis in total darkness. Stimuli were either attached to the chair (head-fixed) or stationary in space and were always well localizable. Surprisingly, spatial updating only occurred when retinal stimulus motion provided sufficient information: long-duration stimuli were always appropriately localized, thus adequately compensating for vestibular nystagmus and the passive head movement during the saccade reaction time. For the shortest stimuli, however, the target was kept in retinocentric coordinates, thus ignoring intervening nystagmus and passive head displacement, regardless of whether the target was moving with the head or not.

Spatial constancy mechanisms in motor control

The success of the human species in interacting with the environment depends on the ability to maintain spatial stability despite the continuous changes in sensory and motor inputs owing to movements of eyes, head and body. In this paper, I will review recent advances in the understanding of how the brain deals with the dynamic flow of sensory and motor information in order to maintain spatial constancy of movement goals. The first part summarizes studies in the saccadic system, showing that spatial constancy is governed by a dynamic feed-forward process, by gaze-centred remapping of target representations in anticipation of and across eye movements. The subsequent sections relate to other oculomotor behaviour, such as eye-head gaze shifts, smooth pursuit and vergence eye movements, and their implications for feed-forward mechanisms for spatial constancy. Work that studied the geometric complexities in spatial constancy and saccadic guidance across head and body movements, distinguishing between self-generated and passively induced motion, indicates that both feed-forward and sensory feedback processing play a role in spatial updating of movement goals. The paper ends with a discussion of the behavioural mechanisms of spatial constancy for arm motor control and their physiological implications for the brain. Taken together, the emerging picture is that the brain computes an evolving representation of three-dimensional action space, whose internal metric is updated in a nonlinear way, by optimally integrating noisy and ambiguous afferent and efferent signals.

Saccadic Compensation for Smooth Eye and Head Movements During Head-Unrestrained Two-Dimensional Tracking

Spatial updating is the ability to keep track of the position of world-fixed objects while we move. In the case of vision, this phenomenon is called spatial constancy and has been studied in head-restraint conditions. During head-restrained smooth pursuit, it has been shown that the saccadic system has access to extraretinal information from the pursuit system to update the objects' position in the surrounding environment. However, during head-unrestrained smooth pursuit, the saccadic system needs to keep track of three different motor commands: the ocular smooth pursuit command, the vestibuloocular reflex (VOR), and the head movement command. The question then arises whether saccades compensate for these movements. To address this question, we briefly presented a target during sinusoidal head-unrestrained smooth pursuit in darkness. Subjects were instructed to look at the flash as soon as they saw it. We observed that subjects were able to orient their gaze to the memorized (and spatially updated) position of the flashed target generally using one to three successive saccades. Similar to the behavior in the head-restrained condition, we found that the longer the gaze saccade latency, the better the compensation for intervening smooth gaze displacements; after about 400 ms, 62% of the smooth gaze displacement had been compensated for. This compensation depended on two independent parameters: the latency of the saccade and the eye contribution to the gaze displacement during this latency period. Separating gaze into eye and head contributions, we show that the larger the eye contribution to the gaze displacement, the better the overall compensation. Finally, we found that the compensation was a function of the head oscillation frequency and we suggest that this relationship is linked to the modulation of VOR gain. We conclude that the general mechanisms of compensation for smooth gaze displacements are similar to those observed in the head-restrained condition.

A vestibular sensation: probabilistic approaches to spatial perception

The vestibular system helps maintain equilibrium and clear vision through reflexes, but it also contributes to spatial perception. In recent years, research in the vestibular field has expanded to higher-level processing involving the cortex. Vestibular contributions to spatial cognition have been difficult to study because the circuits involved are inherently multisensory. Computational methods and the application of Bayes theorem are used to form hypotheses about how information from different sensory modalities is combined together with expectations based on past experience in order to obtain optimal estimates of cognitive variables like current spatial orientation. To test these hypotheses, neuronal populations are being recorded during active tasks in which subjects make decisions based on vestibular and visual or somatosensory information. This review highlights what is currently known about the role of vestibular information in these processes, the computations necessary to obtain the appropriate signals, and the benefits that have emerged thus far.

Spatial updating and the maintenance of visual constancy

Spatial updating is the means by which we keep track of the locations of objects in space even as we move. Four decades of research have shown that humans and non-human primates can take the amplitude and direction of intervening movements into account, including saccades (both head-fixed and head-free), pursuit, whole-body rotations and translations. At the neuronal level, spatial updating is thought to be maintained by receptive field locations that shift with changes in gaze, and evidence for such shifts has been shown in several cortical areas. These regions receive information about the intervening movement from several sources including motor efference copies when a voluntary movement is made and vestibular/somatosensory signals when the body is in motion. Many of these updating signals arise from brainstem regions that monitor our ongoing movements and subsequently transmit this information to the cortex via pathways that likely include the thalamus. Several issues of debate include (1) the relative contribution of extra-retinal sensory and efference copy signals to spatial updating, (2) the source of an updating signal for real life, three-dimensional motion that cannot arise from brain areas encoding only two-dimensional commands, and (3) the reference frames used by the brain to integrate updating signals from various sources. This review highlights the relevant spatial updating studies and provides a summary of the field today. We find that spatial constancy is maintained by a highly evolved neural mechanism that keeps track of our movements, transmits this information to relevant brain regions, and then uses this information to change the way in which single neurons respond. In this way, we are able to keep track of relevant objects in the outside world and interact with them in meaningful ways.

Body-tilt and visual verticality perception during multiple cycles of roll rotation

To assess the effects of degrading canal cues for dynamic spatial orientation in human observers, we tested how judgments about visual-line orientation in space (subjective visual vertical task, SVV) and estimates of instantaneous body tilt (subjective body-tilt task, SBT) develop in the course of three cycles of constant-velocity roll rotation. These abilities were tested across the entire tilt range in separate experiments. For comparison, we also obtained SVV data during static roll tilt. We found that as tilt increased, dynamic SVV responses became strongly biased toward the head pole of the body axis (A-effect), as if body tilt was underestimated. However, on entering the range of near-inverse tilts, SVV responses adopted a bimodal pattern, alternating between A-effects (biased toward head-pole) and E-effects (biased toward feet-pole). Apart from an onset effect, this tilt-dependent pattern of systematic SVV errors repeated itself in subsequent rotation cycles with little sign of worsening performance. Static SVV responses were qualitatively similar and consistent with previous reports but showed smaller A-effects. By contrast, dynamic SBT errors were small and unimodal, indicating that errors in visual-verticality estimates were not caused by errors in body-tilt estimation. We discuss these results in terms of predictions from a canal-otolith interaction model extended with a leaky integrator and an egocentric bias mechanism. We conclude that the egocentric-bias mechanism becomes more manifest during constant velocity roll-rotation and that perceptual errors due to incorrect disambiguation of the otolith signal are small despite the decay of canal signals.

Human visuospatial updating after passive translations in three-dimensional space

To maintain a stable representation of the visual environment as we move, the brain must update the locations of targets in space using extra-retinal signals. Humans can accurately update after intervening active whole-body translations. But can they also update for passive translations (i.e., without efference copy signals of an outgoing motor command)? We asked six head-fixed subjects to remember the location of a briefly flashed target (five possible targets were located at depths of 23, 33, 43, 63, and 150 cm in front of the cyclopean eye) as they moved 10 cm left, right, up, down, forward, or backward while fixating a head-fixed target at 53 cm. After the movement, the subjects made a saccade to the remembered location of the flash with a combination of version and vergence eye movements. We computed an updating ratio where 0 indicates no updating and 1 indicates perfect updating. For lateral and vertical whole-body motion, where updating performance is judged by the size of the version movement, the updating ratios were similar for leftward and rightward translations, averaging 0.84 +/- 0.28 (mean +/- SD) as compared with 0.51 +/- 0.33 for downward and 1.05 +/- 0.50 for upward translations. For forward/backward movements, where updating performance is judged by the size of the vergence movement, the average updating ratio was 1.12 +/- 0.45. Updating ratios tended to be larger for far targets than near targets, although both intra- and intersubject variabilities were smallest for near targets. Thus in addition to self-generated movements, extra-retinal signals involving otolith and proprioceptive cues can also be used for spatial constancy.

Human visuospatial updating after noncommutative rotations

As we move our bodies in space, we often undergo head and body rotations about different axes-yaw, pitch, and roll. The order in which we rotate about these axes is an important factor in determining the final position of our bodies in space because rotations, unlike translations, do not commute. Does our brain keep track of the noncommutativity of rotations when computing changes in head and body orientation and then use this information when planning subsequent motor commands? We used a visuospatial updating task to investigate whether saccades to remembered visual targets are accurate after intervening, whole-body rotational sequences. The sequences were reversed, either yaw then roll or roll then yaw, such that the final required eye movements to reach the same space-fixed target were different in each case. While each subject performed consistently irrespective of target location and rotational combination, we found great intersubject variability in their capacity to update. The distance between the noncommutative endpoints was, on average, half of that predicted by perfect noncommutativity. Nevertheless, most subjects did make eye movements to distinct final endpoint locations and not to one unique location in space as predicted by a commutative model. In addition, their noncommutative performance significantly improved when their less than ideal updating performance was taken into account. Thus the brain can produce movements that are consistent with the processing of noncommutative rotations, although it is often poor in using internal estimates of rotation for updating.

Noncommutative updating of perceived self-orientation in three dimensions

After whole body rotations around an earth-vertical axis in darkness, subjects can indicate their orientation in space with respect to their initial orientation reasonably well. This is possible because the brain is able to mathematically integrate self-velocity information provided by the vestibular system to obtain self-orientation, a process called path integration. For rotations around multiple axes, however, computations are more demanding to accurately update self-orientation with respect to space. In such a case, simple integration is no longer sufficient because of the noncommutativity of rotations. We investigated whether such updating is possible after three-dimensional whole body rotations and whether the noncommutativity of three-dimensional rotations is taken into account. The ability of ten subjects to indicate their spatial orientation in the earth-horizontal plane was tested after different rotational paths from upright to supine positions. Initial and final orientations of the subjects were the same in all cases, but the paths taken were different, and so were the angular velocities sensed by the vestibular system. The results show that seven of the ten subjects could consistently indicate their final orientation within the earth-horizontal plane. Thus perceived final orientation was independent of the path taken, i.e., the noncommutativity of rotations was taken into account.

Deficits and Recovery in Visuospatial Memory During Head Motion After Bilateral Labyrinthine Lesion

To keep a stable internal representation of the environment as we move, extraretinal sensory or motor cues are critical for updating neural maps of visual space. Using a memory-saccade task, we studied whether visuospatial updating uses vestibular information. Specifically, we tested whether trained rhesus monkeys maintain the ability to update the conjugate and vergence components of memory-guided eye movements in response to passive translational or rotational head and body movements after bilateral labyrinthine lesion. We found that lesioned animals were acutely compromised in generating the appropriate horizontal versional responses necessary to update the directional goal of memory-guided eye movements after leftward or rightward rotation/translation. This compromised function recovered in the long term, likely using extravestibular (e.g., somatosensory) signals, such that nearly normal performance was observed 4 mo after the lesion. Animals also lost their ability to adjust memory vergence to account for relative distance changes after motion in depth. Not only were these depth deficits larger than the respective effects on version, but they also showed little recovery. We conclude that intact labyrinthine signals are functionally useful for proper visuospatial memory updating during passive head and body movements.