Neural mechanisms to incorporate visual counterevidence in self-movement estimation

In selecting appropriate behaviors, animals should weigh sensory evidence both for and against specific beliefs about the world. For instance, animals measure optic flow to estimate and control their own rotation. However, existing models of flow detection can be spuriously triggered by visual motion created by objects moving in the world. Here, we show that stationary patterns on the retina, which constitute evidence against observer rotation, suppress inappropriate stabilizing rotational behavior in the fruit fly Drosophila. In silico experiments show that artificial neural networks (ANNs) that are optimized to distinguish observer movement from external object motion similarly detect stationarity and incorporate negative evidence. Employing neural measurements and genetic manipulations, we identified components of the circuitry for stationary pattern detection, which runs parallel to the fly's local motion and optic-flow detectors. Our results show how the fly brain incorporates negative evidence to improve heading stability, exemplifying how a compact brain exploits geometrical constraints of the visual world.

Vection induced by a pair of patches of synchronized visual motion stimuli covering total field of views as small as 10 square-degrees

Vection (illusion of self-motion) is known to be induced by watching large field-of-view (FOV) moving scenes. In our study, we investigated vection induced by small FOV stimuli. Three experiments were conducted in 45 sessions to analyze vection provoked by moving scenes covering total FOVs as small as 10 square-degrees. Results indicated that 88% of the participants reported vection while watching two small patches of moving dots (1° horizontal by 5° vertical, each) placed on the left and right sides of the observers. This is less than a quarter of the total visual area of two Apple Watches viewed at a distance of 40 cm. Occlusion of the visual field between the two display patches significantly increased the levels of rated vection. Similarly, increasing the speed of the moving dots of the two display patches from about 5 to 25 °/sec increased the levels of rated vection significantly. The location of the two patches in the horizontal visual field did not affect the vection perception significantly. When the two straight stripes of dots were moving in opposite directions, participants perceived circular vection. The observers connected the two stimuli in their minds and perceived them as parts of a single occluded background. The findings of this study are relevant to the design of mobile devices (e.g., smartphones) and wearable technology (e.g., smart watches) with small display areas.

Self-initiation Inhibits the Postural and Electrophysiological Responses to Optic Flow and Button Pressing

As we move through our environment, our visual system is presented with optic flow, a potentially important cue for perception, navigation and postural control. How does the brain anticipate the optic flow that arises as a consequence of our own movement? Converging evidence suggests that stimuli are processed differently by the brain if occurring as a consequence of self-initiated actions, compared to when externally generated. However, this has mainly been demonstrated with auditory stimuli. It is not clear how this occurs with optic flow. We measured behavioural, neurophysiological and head motion responses of 29 healthy participants to radially expanding, vection-inducing optic flow stimuli, simulating forward transitional motion, which were either initiated by the participant's own button-press ("self-initiated flow") or by the computer ("passive flow"). Self-initiation led to a prominent and left-lateralized inhibition of the flow-evoked posterior event-related alpha desynchronization (ERD), and a stabilisation of postural responses. Neither effect was present in control button-press-only trials, without optic flow. Additionally, self-initiation also produced a large event-related potential (ERP) negativity between 130-170 ms after optic flow onset. Furthermore, participants' visual induced motion sickness (VIMS) and vection intensity ratings correlated positively across the group - although many participants felt vection in the absence of any VIMS, none reported the opposite combination. Finally, we found that the simple act of making a button press leads to a detectable head movement even when using a chin rest. Taken together, our results indicate that the visual system is capable of predicting optic flow when self-initiated, to affect behaviour.

Hot Wind to the Body Can Facilitate Vection Only When Participants Walk Through a Fire Corridor Virtually

Vection has been reported to be enhanced by wind, as long as the wind is a normal temperature and not hot. However, here we report that a hot wind can facilitate vection, as long as it is natural and consistent with the visual stimulus. We created a fire-corridor stimulus that was consistent with a hot wind and a control stimulus composed of cubes, which were irrelevant to a hot wind. We compared the vection strength induced by a fire-corridor (fire condition) visual stimulus with that induced by geometric cubes (no-fire condition) visual stimulus. There were three wind type conditions: a normal temperature wind, hot wind, and no wind. The results showed that a normal temperature wind facilitated vection and that a hot wind (but not a normal wind) highly enhanced vection when a fire-corridor stimulus was presented. These results suggest that vection is highly affected and modulated by high-level cognitive processes.

Different Head-Sway Responses to Optic Flow in Sitting and Standing With a Head-Mounted Display

We investigated postural responses (head displacements) and self-motion perception (vection) to radial and lateral optic flows while sitting and standing by using a head-mounted display. We found that head displacement directions varied across postures. In the standing posture, radial optic flow generally produced the opposed head displacement against the perceived vection direction, consistent with the literature; however, in the sitting posture, the optic flow generally produced the following head displacement in the vection direction. In the standing posture, responses were evident soon after the onset of the optic flow presentation but became less clear in the latter half of a trial. The results, while less clear for lateral flows, were similar for both flow types. Our findings suggest partially distinct processes underlying vection and postural control.

Differences in Three Vection Indices (Latency, Duration, and Magnitude) Induced by "Camera-Moving" and "Object-Moving" in a Virtual Computer Graphics World, Despite Similarity in the Retinal Images

To create a self-motion (vection) situation in three-dimensional computer graphics (CG), there are mainly two ways: moving a camera toward an object ("camera moving") or by moving the object and its surrounding environment toward the camera ("object moving"). As both methods vary considerably in the amount of computer calculations involved in generating CG, knowing how each method affects self-motion perception should be important to CG-creators and psychologists. Here, we simulated self-motion in a virtual three-dimensional CG-world, without stereoscopic disparity, which correctly reflected the lighting and glare. Self-motion was induced by "camera moving" or by "object moving," which in the present experiments was done by moving a tunnel surrounding the camera toward the camera. This produced two retinal images that were virtually identical in Experiment 1 and very similar in Experiments 2 and 3. The stimuli were presented on a large plasma display to 15 naive participants and induced substantial vection. Three experiments comparing vection strength between the two methods found weak but significant differences. The results suggest that when creating CG visual experiences, "camera-moving" induces stronger vection.

Effects of Body Orientation Relative to Gravity on Vection in Children and Adults

We investigated the effects of the interaction between the body and gravitational axes on vection (visually induced self-motion perception) in school-age children and adults. Experiment 1 was a pilot study of adults that was conducted to determine the appropriate experimental settings for the main experiment that included children and adults. The adult participants experienced vection in four different directions in the head-centered coordinate (forward, backward, upward, and downward) under two postural conditions: standing (in which the body and gravitational axes were consistent) and supine (in which the body orientation was orthogonally aligned to the gravitational axis). The adults reported more rapid and longer lasting vection when standing than when supine. In the main experiment (Experiment 2), we tested adults and school-age children under conditions similar to those of Experiment 1 and found that the reported vection was more rapid and longer lasting in children than in adults, whereas the reported vection tended to be more rapid and longer lasting under the standing condition than the supine condition for both age groups. Based on the similarities and differences between children and adults found in the present and previous vection studies, child-specific features of vection are discussed.

Modulation of alpha waves in sensorimotor cortical networks during self-motion perception evoked by different visual-vestibular conflicts

Visually induced illusion of self-motion (vection) has been used as a tool to address neural correlates of visual-vestibular interaction. The extent to which vestibular cortical areas are deactivated during vection varies from one study to another. The main question in this study is whether such deactivation depends on the visual-vestibular conflict induced by visual motion. A visual motion about the line of sight (roll motion) induces a visual-canal conflict in upright and supine observers. An additional visual-otolith conflict arises in the upright position only, with the graviceptive inputs indicating that the head is stationary. A 96-channel electroencephalogram (EEG) was recorded in 21 participants exposed to roll motion in seated and supine positions. Meanwhile, perceptual state of self-motion was recorded. Results showed a transient decrease in the cortical sensorimotor networks' alpha activity at the onset of vection whatever the participant's position, and therefore the visual-vestibular conflict. During vection, an increase in alpha activity over parieto-occipital areas was observed in the upright condition, that is, in a condition of visual-otolith conflict. The modulation of alpha activity may be predictive of the illusion of self-motion but also may reflect the level of inhibition in the sensorimotor networks needed to reduce potential interference from vestibular conflicting inputs.NEW & NOTEWORTHY For the first time, we explored the neural correlates of different visuo-vestibular conflicts induced by visual motion using EEG. Our study highlighted a neuronal signature for illusory self-motion (vection) in the sensorimotor networks. Strong alpha activity may predict successful vection but also reflects the level of inhibition of sensorimotor networks needed to reduce potential interfering vestibular inputs. These findings would be of prime importance for simulator and virtual reality systems that induce vection.

The Effect of Optical Flow Motion Direction on Vection Strength

In some phenomena of visual perception, the motion direction of visual stimuli can affect perception. In particular, asymmetries between oblique directions and cardinal (horizontal and vertical) directions have been reported and are known as oblique effects (e.g., contrast sensitivity and motion threshold). In this study, we investigated how vection strength depends on motion direction. Participants observed random-dot optical flow in a circular field and rated the perceived vection strength. Dot movement was systematically controlled using the following angles: 0° (up), 30°, 45°, 60°, 90°, 120°, 135°, 150°, and 180° (down). We found that vection strength depended on motion direction and was weaker in the oblique directions than cardinal directions. Thus, the effect of motion direction on vection strength was variable, as seen in the shape of the oblique effect.

Larger Head Displacement to Optic Flow Presented in the Lower Visual Field

Optic flow that simulates self-motion often produces postural adjustment. Although literature has suggested that human postural control depends largely on visual inputs from the lower field in the environment, effects of the vertical location of optic flow on postural responses are not well investigated. Here, we examined whether optic flow presented in the lower visual field produces stronger responses than optic flow in the upper visual field. Either expanding or contracting optic flow was presented in upper, lower, or full visual fields through an Oculus Rift head-mounted display. Head displacement and vection strength were measured. Results showed larger head displacement under the optic flow presentation in the full visual field and the lower visual field than the upper visual field, during early period of presentation of the contracting optic flow. Vection was strongest in the full visual field and weakest in the upper visual field. Our findings of lower field superiority in head displacement and vection support the notion that ecologically relevant information has a particularly important role in human postural control and self-motion perception.

Virtual Walking Sensation by Prerecorded Oscillating Optic Flow and Synchronous Foot Vibration

This article reports the first psychological evidence that the combination of oscillating optic flow and synchronous foot vibration evokes a walking sensation. In this study, we first captured a walker's first-person-view scenes with footstep timings. Participants observed the naturally oscillating scenes on a head-mounted display with vibrations on their feet and rated walking-related sensations using a Visual Analogue Scale. They perceived stronger sensations of self-motion, walking, leg action, and telepresence from the oscillating visual flow with foot vibrations than with randomized-timing vibrations or without vibrations. The artificial delay of foot vibrations with respect to the scenes diminished the walking-related sensations. These results suggest that the oscillating visual scenes and synchronous foot vibrations are effective for creating virtual walking sensations.

Shift in lateralization during illusory self-motion: EEG responses to visual flicker at 10 Hz and frequency-specific modulation by tACS

Self-motion perception is a key aspect of higher vestibular processing, suggested to rely upon hemispheric lateralization and alpha-band oscillations. The first aim of this study was to test for any lateralization in the EEG alpha band during the illusory sense of self-movement (vection) induced by large optic flow stimuli. Visual stimuli flickered at alpha frequency (approx. 10 Hz) in order to produce steady state visually evoked potentials (SSVEPs), a robust EEG measure which allows probing the frequency-specific response of the cortex. The first main result was that differential lateralization of the alpha SSVEP response was found during vection compared with a matched random motion control condition, supporting the idea of lateralization of visual-vestibular function. Additionally, this effect was frequency-specific, not evident with lower frequency SSVEPs. The second aim of this study was to test for a causal role of the right hemisphere in producing this lateralization effect and to explore the possibility of selectively modulating the SSVEP response. Transcranial alternating current stimulation (tACS) was applied over the right hemisphere simultaneously with SSVEP recording, using a novel artefact removal strategy for combined tACS-EEG. The second main result was that tACS enhanced SSVEP amplitudes, and the effect of tACS was not confined to the right hemisphere. Subsequent control experiments showed the effect of tACS requires the flicker frequency and tACS frequency to be closely matched and tACS to be of sufficient intensity. Combined tACS-SSVEPs are a promising method for future investigation into the role of neural oscillations and for optimizing tACS.

Sensitivity to Visual Speed Modulation in Head-Mounted Displays Depends on Fixation

A primary cause of simulator sickness in head-mounted displays (HMDs) is conflict between the visual scene displayed to the user and the visual scene expected by the brain when the user's head is in motion. It is useful to measure perceptual sensitivity to visual speed modulation in HMDs because conditions that minimize this sensitivity may prove less likely to elicit simulator sickness. In prior research, we measured sensitivity to visual gain modulation during slow, passive, full-body yaw rotations and observed that sensitivity was reduced when subjects fixated a head-fixed target compared with when they fixated a scene-fixed target. In the current study, we investigated whether this pattern of results persists when (1) movements are faster, active head turns, and (2) visual stimuli are presented on an HMD rather than on a monitor. Subjects wore an Oculus Rift CV1 HMD and viewed a 3D scene of white points on a black background. On each trial, subjects moved their head from a central position to face a 15° eccentric target. During the head movement they fixated a point that was either head-fixed or scene-fixed, depending on condition. They then reported if the visual scene motion was too fast or too slow. Visual speed on subsequent trials was modulated according to a staircase procedure to find the speed increment that was just noticeable. Sensitivity to speed modulation during active head movement was reduced during head-fixed fixation, similar to what we observed during passive whole-body rotation. We conclude that fixation of a head-fixed target is an effective way to reduce sensitivity to visual speed modulation in HMDs, and may also be an effective strategy to reduce susceptibility to simulator sickness.

The A-Effect and Global Motion

When the head is tilted, an objectively vertical line viewed in isolation is typically perceived as tilted. We explored whether this shift also occurs when viewing global motion displays perceived as either object-motion or self-motion. Observers stood and lay left side down while viewing (1) a static line, (2) a random-dot display of 2-D (planar) motion or (3) a random-dot display of 3-D (volumetric) global motion. On each trial, the line orientation or motion direction were tilted from the gravitational vertical and observers indicated whether the tilt was clockwise or counter-clockwise from the perceived vertical. Psychometric functions were fit to the data and shifts in the point of subjective verticality (PSV) were measured. When the whole body was tilted, the perceived tilt of both a static line and the direction of optic flow were biased in the direction of the body tilt, demonstrating the so-called A-effect. However, we found significantly larger shifts for the static line than volumetric global motion as well as larger shifts for volumetric displays than planar displays. The A-effect was larger when the motion was experienced as self-motion compared to when it was experienced as object-motion. Discrimination thresholds were also more precise in the self-motion compared to object-motion conditions. Different magnitude A-effects for the line and motion conditions—and for object and self-motion—may be due to differences in combining of idiotropic (body) and vestibular signals, particularly so in the case of vection which occurs despite visual-vestibular conflict.

Estimating the sensorimotor components of cybersickness

The user base of the virtual reality (VR) medium is growing, and many of these users will experience cybersickness. Accounting for the vast interindividual variability in cybersickness forms a pivotal step in solving the issue. Most studies of cybersickness focus on a single factor (e.g., balance, sex, or vection), while other contributors are overlooked. Here, we characterize the complex relationship between cybersickness and several measures of sensorimotor processing. In a single session, we conducted a battery of tests of balance control, vection responses, and vestibular sensitivity to self-motion. Following this, we measured cybersickness after VR exposure. We constructed a principal components regression model using the measures of sensorimotor processing. The model significantly predicted 37% of the variability in cybersickness measures, with 16% of this variance being accounted for by a principal component that represented balance control measures. The strongest predictor was participants' sway path length during vection, which was inversely related to cybersickness [ r(28) = -0.53, P = 0.002] and uniquely accounted for 7.5% of the variance in cybersickness scores across participants. Vection strength reports and measures of vestibular sensitivity were not significant predictors of cybersickness. We discuss the possible role of sensory reweighting in cybersickness that is suggested by these results, and we identify other factors that may account for the remaining variance in cybersickness. The results reiterate that the relationship between balance control and cybersickness is anything but straightforward. NEW & NOTEWORTHY The advent of consumer virtual reality provides a pressing need for interventions that combat sickness in simulated environments (cybersickness). This research builds on multiple theories of cybersickness etiology to develop a predictive model that distinguishes between individuals who are/are not likely to experience cybersickness. In the future this approach can be adapted to provide virtual reality users with curated content recommendations based on more efficient measurements of sensorimotor processing.

Allocating less attention to central vision during vection is correlated with less motion sickness

Visually induced motion sickness (VIMS) is a common discomfort response associated with vection-provoking stimuli. It has been suggested that susceptibility to VIMS depends on the ability to regulate visual performance during vection. To test this, 29 participants, with VIMS susceptibility assessed by Motion Sickness Susceptibility Questionnaire, were recruited to undergo three series of sustained attention to response tests (SARTs) while watching dot pattern stimuli known to provoke roll-vection. In general, SARTs performance was impaired in the central visual field (CVF), but improved in peripheral visual field (PVF), suggesting the reallocation of attention during vection. Moreover, VIMS susceptibility was negatively correlated with the effect sizes, suggesting that participants who were less susceptible to VIMS showed better performance in attention re-allocation. Finally, when trained to re-allocation attention from the CVF to the PVF, participants experienced more stable vection. Findings provide a better understanding of VIMS and shed light on possible preventive measures. Practitioner Summary: Allocating less visual attention to central visual field during visual motion stimulation is associated with stronger vection and higher resistance to motion sickness. Virtual reality application designers may utilise the location of visual tasks to strengthen and stabilise vection, while reducing the potential of visually induced motion sickness.

Vection Is Enhanced by Increased Exposure to Optic Flow

We examined whether vection strength could be modulated by altering the exposure duration to optic flow. Experiment 1 sourced 150 different video clips from various Japanese animation works which simulated self-motion. Despite large differences in the content of these video clips, we found a significant positive correlation between their play durations and their ratings of vection magnitude. Experiment 2 examined this relationship further using more tightly controlled visual motion stimuli. Vection was induced by presenting the motion of the same expanding grating stimulus for 8, 16, 32, or 64 seconds. While vection onset latencies remained constant across these four conditions, vection magnitude/strength was found to increase systematically with the exposure duration. As predicted by a recent computational model of vection, we conclude that subjective vection strength does depend on the exposure duration to optic flow.

Development of Asymmetric Vection for Radial Expansion or Contraction Motion: Comparison Between School-Age Children and Adults

Vection is illusory self-motion elicited by visual stimuli and is more easily induced by radial contraction than expansion flow in adults. The asymmetric feature of vection was reexamined with 18 younger (age: 6–8 years) and 19 older children (age: 9–11 years) and 20 adults. In each experimental trial, participants observed either radial expansion or contraction flow; the latency, cumulative duration, and saturation of vection were measured. The results indicated that the latency for contraction was significantly shorter than that for expansion in all age-groups. In addition, the latency and saturation were significantly shorter and greater, respectively, in the younger or older children compared with the adults, regardless of the flow pattern. These results indicate that the asymmetry in vection for expansion or contraction flow emerges by school age, and that school-age children experience significantly more rapid and stronger vection than adults.

The Oscillating Potential Model of Visually Induced Vection

Visually induced illusions of self-motion are often referred to as vection. This article developed and tested a model of responding to visually induced vection. We first constructed a mathematical model based on well-documented characteristics of vection and human behavioral responses to this illusion. We then conducted 10,000 virtual trial simulations using this Oscillating Potential Vection Model (OPVM). OPVM was used to generate simulated vection onset, duration, and magnitude responses for each of these trials. Finally, we compared the properties of OPVM's simulated vection responses with real responses obtained in seven different laboratory-based vection experiments. The OPVM output was found to compare favorably with the empirically obtained vection data.

Identifying Objective EEG Based Markers of Linear Vection in Depth

This proof-of-concept study investigated whether a time-frequency EEG approach could be used to examine vection (i.e., illusions of self-motion). In the main experiment, we compared the event-related spectral perturbation (ERSP) data of 10 observers during and directly after repeated exposures to two different types of optic flow display (each was 35° wide by 29° high and provided 20 s of motion stimulation). Displays consisted of either a vection display (which simulated constant velocity forward self-motion in depth) or a control display (a spatially scrambled version of the vection display). ERSP data were decomposed using time-frequency Principal Components Analysis (t–f PCA). We found an increase in 10 Hz alpha activity, peaking some 14 s after display motion commenced, which was positively associated with stronger vection ratings. This followed decreases in beta activity, and was also followed by a decrease in delta activity; these decreases in EEG amplitudes were negatively related to the intensity of the vection experience. After display motion ceased, a series of increases in the alpha band also correlated with vection intensity, and appear to reflect vection- and/or motion-aftereffects, as well as later cognitive preparation for reporting the strength of the vection experience. Overall, these findings provide support for the notion that EEG can be used to provide objective markers of changes in both vection status (i.e., “vection/no vection”) and vection strength.

Relative Visual Oscillation Can Facilitate Visually Induced Self-Motion Perception

Adding simulated viewpoint jitter or oscillation to displays enhances visually induced illusions of self-motion (vection). The cause of this enhancement is yet to be fully understood. Here, we conducted psychophysical experiments to investigate the effects of different types of simulated oscillation on vertical vection. Observers viewed horizontally oscillating and nonoscillating optic flow fields simulating downward self-motion through an aperture. The aperture was visually simulated to be nearer to the observer and was stationary or oscillating in-phase or counter-phase to the direction of background horizontal oscillations of optic flow. Results showed that vection strength was modulated by the oscillation of the aperture relative to the background optic flow. Vertical vection strength increased as the relative oscillatory horizontal motion between the flow and the aperture increased. However, such increases in vection were only generated when the added oscillations were orthogonal to the principal direction of the optic flow pattern, and not when they occurred in the same direction. The oscillation effects observed in this investigation could not be explained by motion adaptation or different (motion parallax based) effects on depth perception. Instead, these results suggest that the oscillation advantage for vection depends on relative visual motion.

A New Angle on Object-Background Effects in Vection

We considered whether optic flow generated by 3D relief of a foreground surface might influence visually-mediated self-motion perception (vection). We generated background motion consistent with self-rotation, and a foreground object with bumpy relief was either rotated with the observer (ego-centric) or fixed in world coordinates (world-centric). We found that vection strength ratings were greater in conditions with world-centric retinal motion of the foreground object, despite generating flow that was opposite to background motion. This effect was explained by observer judgments of the axis self-rotation in depth; whereas ego-centric flow generated experiences of more on-axis self-rotation, world-centric flow generated experiences of centrifugal rotation around the foreground object. These data suggest that foreground object motion can increase the perception of self-motion generated by optic flow, even when they reduce net retinal motion coherence and promote conditions for multisensory conflict. This finding supports the view that self-motion perception depends on mid-level representations of whole-scene motion.

More than a cool illusion? Functional significance of self-motion illusion (circular vection) for perspective switches

Self-motion can facilitate perspective switches and “automatic spatial updating” and help reduce disorientation in applications like virtual reality (VR). However, providing physical motion through moving-base motion simulators or free-space walking areas comes with high cost and technical complexity. This study provides first evidence that merely experiencing an embodied illusion of self-motion (“circular vection”) can provide similar behavioral benefits as actual self-motion: Blindfolded participants were asked to imagine facing new perspectives in a well-learned room, and point to previously learned objects. Merely imagining perspective switches while stationary yielded worst performance. When perceiving illusory self-rotation to the novel perspective, however, performance improved significantly and yielded performance similar to actual rotation. Circular vection was induced by combining rotating sound fields (“auditory vection”) and biomechanical vection from stepping along a carrousel-like rotating floor platter. In sum, illusory self-motion indeed facilitated perspective switches and thus spatial orientation, similar to actual self-motion, thus providing first compelling evidence of the functional significance and behavioral relevance of vection. This could ultimately enable us to complement the prevailing introspective vection measures with behavioral indicators, and guide the design for more affordable yet effective VR simulators that intelligently employ multi-modal self-motion illusions to reduce the need for costly physical observer motion.

Effects of auditory information on self-motion perception during simultaneous presentation of visual shearing motion

Recent studies have found that self-motion perception induced by simultaneous presentation of visual and auditory motion is facilitated when the directions of visual and auditory motion stimuli are identical. They did not, however, examine possible contributions of auditory motion information for determining direction of self-motion perception. To examine this, a visual stimulus projected on a hemisphere screen and an auditory stimulus presented through headphones were presented separately or simultaneously, depending on experimental conditions. The participant continuously indicated the direction and strength of self-motion during the 130-s experimental trial. When the visual stimulus with a horizontal shearing rotation and the auditory stimulus with a horizontal one-directional rotation were presented simultaneously, the duration and strength of self-motion perceived in the opposite direction of the auditory rotation stimulus were significantly longer and stronger than those perceived in the same direction of the auditory rotation stimulus. However, the auditory stimulus alone could not sufficiently induce self-motion perception, and if it did, its direction was not consistent within each experimental trial. We concluded that auditory motion information can determine perceived direction of self-motion during simultaneous presentation of visual and auditory motion information, at least when visual stimuli moved in opposing directions (around the yaw-axis). We speculate that the contribution of auditory information depends on the plausibility and information balance of visual and auditory information.

The effects of local rotation on roll vection induced by globally rotating visual inducer

A visual stimulus rotating globally along an observer's line of sight can induce the illusory perception of self-rotation in the opposite direction (roll vection). Psychophysical experiments were conducted to examine the effects of local rotations of visual elements of the stimulus that were manipulated independently of the global rotation. The results indicated that the addition of local rotations inconsistent with the global rotation (assumed to be the primary inducer of roll vection), generally decreased the strength of perceived self-rotation. The uniformity of orientation of the elements composing the global visual pattern and the visual polarities assigned to each visual element, i.e., intrinsic directionality concerning up and down, were observed to function as modulators of the effects of the local rotation. These results suggested that local motion signals arising from independent rotations assigned to each element of a visual object cannot be ignored in the perceptual mechanism underlying roll vection.

Optic-flow selective cortical sensory regions associated with self-reported states of vection

Optic flow is one of the most important visual cues to the estimation of self-motion. It has repeatedly been demonstrated that a cortical network including visual, multisensory, and vestibular areas is implicated in processing optic flow; namely, visual areas middle temporal cortex (MT+), V6; multisensory areas ventral intra-parietal area (VIP), cingulate sulcus visual area, precuneus motion area (PcM); and vestibular areas parieto-insular vestibular cortex (PIVC) and putative area 2v (p2v). However, few studies have investigated the roles of and interaction between the optic-flow selective sensory areas within the context of self-motion perception. When visual information (i.e., optic flow) is the sole cue to computing self-motion parameters, the discrepancy amongst the sensory signals may induce an illusion of self-motion referred to as ‘vection.’ This study aimed to identify optic-flow selective sensory areas that are involved in the processing of visual cues to self-motion, by introducing vection as an index and assessing activation in which of those areas reflect vection, using functional magnetic resonance imaging. The results showed that activity in visual areas MT+ and V6, multisensory area VIP and vestibular area PIVC was significantly greater while participants were experiencing vection, as compared to when they were experiencing no vection, which may indicate that activation in MT+, V6, VIP, and PIVC reflects vection. The results also place VIP in a good position to integrate visual cues related to self-motion and vestibular information.

Comparing the effectiveness of different displays in enhancing illusions of self-movement (vection)

Illusions of self-movement (vection) can be used in virtual reality (VR) and other applications to give users the embodied sensation that they are moving when physical movement is unfeasible or too costly. Whereas a large body of vection literature studied how various parameters of the presented visual stimulus affect vection, little is known how different display types might affect vection. As a step toward addressing this gap, we conducted three experiments to compare vection and usability parameters between commonly used VR displays, ranging from stereoscopic projection and 3D TV to high-end head-mounted display (HMD, NVIS SX111) and recent low-cost HMD (Oculus Rift). The last experiment also compared these two HMDs in their native full field of view (FOV) and a reduced, matched FOV of 72° × 45°. Participants moved along linear and curvilinear paths in the virtual environment, reported vection onset time, and rated vection intensity at the end of each trial. In addition, user ratings on immersion, motion sickness, vection, and overall preference were recorded retrospectively and compared between displays. Unexpectedly, there were no significant effects of display on vection measures. Reducing the FOV for the HMDs (from full to 72° × 45°) decreased vection onset latencies, but did not affect vection intensity. As predicted, curvilinear paths yielded earlier and more intense vection. Although vection has often been proposed to predict or even cause motion sickness, we observed no correlation for any of the displays studied. In conclusion, perceived self-motion and other user experience measures proved surprisingly tolerant toward changes in display type as long as the FOV was roughly matched. This suggests that display choice for vection research and VR applications can be largely based on other considerations as long as the provided FOV is sufficiently large.

Single stimulus color can modulate vection

In the present study, we investigated the effects of single color on forward and backward vection. The approaching or receding optical flow observed during forward or backward locomotion was simulated by using random dots with changing size, velocity, and disparity. The dots were presented on a black (Experiments 1 and 2) or white background (Experiment 3) in equiluminant colors; namely, white (or gray), red, yellow, green, or blue. The participant's task was to press and hold one of three buttons whenever they felt vection. The three buttons corresponded to the subjective strength of vection: strong, same, and weak relative to vection induced by the standard modulus. In Experiments 1 and 2, the participants were also asked to rate the strength and direction of vection after each trial. In Experiment 3, they rated the visibility and the perceived velocity of dot motion. Experiment 1 showed that the induced vection was stronger for the chromatic than for the achromatic dots. Particularly at low velocity conditions (±10 km/h), the vection induced for red dots was stronger than that for the other colored dots. Experiment 2 showed that the order effects of stimulus presentation could not explain the findings of Experiment 1. Experiment 3's pattern of results was similar to that of Experiment 1, and this suggested that a luminance artifact between color conditions could not account for Experiment 1's findings. These results suggest that a stimulus color can modulate vection even when a single color is added to the optical flow.

Vection and visually induced motion sickness: how are they related?

The occurrence of visually induced motion sickness has been frequently linked to the sensation of illusory self-motion (vection), however, the precise nature of this relationship is still not fully understood. To date, it is still a matter of debate as to whether vection is a necessary prerequisite for visually induced motion sickness (VIMS). That is, can there be VIMS without any sensation of self-motion? In this paper, we will describe the possible nature of this relationship, review the literature that addresses this relationship (including theoretical accounts of vection and VIMS), and offer suggestions with respect to operationally defining and reporting these phenomena in future.

The Oculus Rift: a cost-effective tool for studying visual-vestibular interactions in self-motion perception

For years now, virtual reality devices have been applied in the field of vision science in an attempt to improve our understanding of perceptual principles underlying the experience of self-motion. Some of this research has been concerned with exploring factors involved in the visually-induced illusory perception of self-motion, known as vection. We examined the usefulness of the cost-effective Oculus Rift in generating vection in seated observers. This device has the capacity to display optic flow in world coordinates by compensating for tracked changes in 3D head orientation. We measured vection strength in three conditions of visual compensation for head movement: compensated, uncompensated, and inversely compensated. During presentation of optic flow, the observer was instructed to make periodic head oscillations (±22° horizontal excursions at approximately 0.53 Hz). We found that vection was best in the compensated condition, and was weakest in the inversely compensated condition. Surprisingly, vection was always better in passive viewing conditions, compared with conditions where active head rotations were performed. These findings suggest that vection is highly dependent on interactions between visual, vestibular and proprioceptive information, and may be highly sensitive to limitations of temporal lag in visual-vestibular coupling using this system.

The mirror illusion: does proprioceptive drift go hand in hand with sense of agency?

Vection can be regarded as the illusion of “whole-body” position perception. In contrast, the mirror illusion is that of “body-part” position perception. When participants viewed their left hands in a mirror positioned along the midsaggital axis while moving both hands synchronously, they hardly noticed the spatial offset between the hand in the mirror and the obscured real right hand. This illusion encompasses two phenomena: proprioceptive drift and sense of agency. Proprioceptive drift represented a perceptual change in the position of the obscured hand relative to that of the hand in the mirror. Sense of agency referred to the participants' subjective sense of controlling body image as they would their own bodies. We examined the spatial offset between these two phenomena. Participants responded to a two-alternative forced choice (2AFC) question regarding the subjective position of their right hands and questionnaires regarding sense of agency at various positions of the right hand. We analyzed the 2AFC data using a support vector machine and compared its classification result and the questionnaire results. Our data analysis suggested that the two phenomena were observed in concentric space, but the estimated range of the proprioceptive drift was slightly narrower than the range of agency. Although this outcome can be attributed to differences in measurement or analysis, to our knowledge, this is the first report to suggest that proprioceptive drift and sense of agency are concentric and almost overlap.

Future challenges for vection research: definitions, functional significance, measures, and neural bases

This paper discusses four major challenges facing modern vection research. Challenge 1 (Defining Vection) outlines the different ways that vection has been defined in the literature and discusses their theoretical and experimental ramifications. The term vection is most often used to refer to visual illusions of self-motion induced in stationary observers (by moving, or simulating the motion of, the surrounding environment). However, vection is increasingly being used to also refer to non-visual illusions of self-motion, visually mediated self-motion perceptions, and even general subjective experiences (i.e., “feelings”) of self-motion. The common thread in all of these definitions is the conscious subjective experience of self-motion. Thus, Challenge 2 (Significance of Vection) tackles the crucial issue of whether such conscious experiences actually serve functional roles during self-motion (e.g., in terms of controlling or guiding the self-motion). After more than 100 years of vection research there has been surprisingly little investigation into its functional significance. Challenge 3 (Vection Measures) discusses the difficulties with existing subjective self-report measures of vection (particularly in the context of contemporary research), and proposes several more objective measures of vection based on recent empirical findings. Finally, Challenge 4 (Neural Basis) reviews the recent neuroimaging literature examining the neural basis of vection and discusses the hurdles still facing these investigations.

Can walking motions improve visually induced rotational self-motion illusions in virtual reality?

Illusions of self-motion (vection) can provide compelling sensations of moving through virtual environments without the need for complex motion simulators or large tracked physical walking spaces. Here we explore the interaction between biomechanical cues (stepping along a rotating circular treadmill) and visual cues (viewing simulated self-rotation) for providing stationary users a compelling sensation of rotational self-motion (circular vection). When tested individually, biomechanical and visual cues were similarly effective in eliciting self-motion illusions. However, in combination they yielded significantly more intense self-motion illusions. These findings provide the first compelling evidence that walking motions can be used to significantly enhance visually induced rotational self-motion perception in virtual environments (and vice versa) without having to provide for physical self-motion or motion platforms. This is noteworthy, as linear treadmills have been found to actually impair visually induced translational self-motion perception (Ash, Palmisano, Apthorp, & Allison, 2013). Given the predominant focus on linear walking interfaces for virtual-reality locomotion, our findings suggest that investigating circular and curvilinear walking interfaces offers a promising direction for future research and development and can help to enhance self-motion illusions, presence and immersion in virtual-reality systems.

Evidence against an ecological explanation of the jitter advantage for vection

Visual-vestibular conflicts have been traditionally used to explain both perceptions of self-motion and experiences of motion sickness. However, sensory conflict theories have been challenged by findings that adding simulated viewpoint jitter to inducing displays enhances (rather than reduces or destroys) visual illusions of self-motion experienced by stationary observers. One possible explanation of this jitter advantage for vection is that jittering optic flows are more ecological than smooth displays. Despite the intuitive appeal of this idea, it has proven difficult to test. Here we compared subjective experiences generated by jittering and smooth radial flows when observers were exposed to either visual-only or multisensory self-motion stimulations. The display jitter (if present) was generated in real-time by updating the virtual computer-graphics camera position to match the observer’s tracked head motions when treadmill walking or walking in place, or was a playback of these head motions when standing still. As expected, the (more naturalistic) treadmill walking and the (less naturalistic) walking in place were found to generate very different physical head jitters. However, contrary to the ecological account of the phenomenon, playbacks of treadmill walking and walking in place display jitter both enhanced visually induced illusions of self-motion to a similar degree (compared to smooth displays).

Surface qualities have little effect on vection strength

We investigated the effects of different surface qualities of materials on vection strength. Previous studies have extensively examined the stimulus parameters for effective vection induction. However, the effects of surface qualities on vection induction have not been studied at all despite their importance in realistic perception of a scene. As a first step toward understanding the effects of surface qualities on vection, we investigated surface qualities derived from light-reflecting properties of nine material categories commonly encountered in daily life: bark, ceramic, fabric, fur, glass, leather, metal, stone and wood. To relate vection strength with low-level visual features and with subjective impression of materials, we analyzed spatial frequency and participants' ratings of adjective pairs that describe impressions of material categories. Although the nine material categories were perceived differently, there was no main effect of material condition on vection strength. However, multiple regression analyses revealed that vection was partially explained by both spatial frequency and principal components extracted from the subjective impression. These results indicate that although the effect of surface qualities of materials on vection is small, both low-level image-based and perceptual-level processing of surface qualities may influence vection¹.

Stronger vection in junior high school children than in adults

Previous studies have shown that even elementary school-aged children (7 and 11 years old) experience visually induced perception of illusory self-motion (vection) (Lepecq et al., 1995, Perception, 24, 435–449) and that children of a similar age (mean age = 9.2 years) experience more rapid and stronger vection than do adults (Shirai et al., 2012, Perception, 41, 1399–1402). These findings imply that although elementary school-aged children experience vection, this ability is subject to further development. To examine the subsequent development of vection, we compared junior high school students' (N = 11, mean age = 14.4 years) and adults' (N = 10, mean age = 22.2 years) experiences of vection. Junior high school students reported significantly stronger vection than did adults, suggesting that the perceptual experience of junior high school students differs from that of adults with regard to vection and that this ability undergoes gradual changes over a relatively long period of development.

A new spin on vection in depth

Previous research has shown that adding lateral viewpoint changes to visual displays simulating self-motion in depth can increase the strength of linear vection. We performed experiments to determine whether these vection increases are caused by reduced adaptation to retinal motion, rather than increased motion parallax in the visual display. In Experiment 1, we added increasing amplitudes of sinusoidal angular viewpoint oscillation around the viewing axis (up to 94.2°/s) to radial flow simulating self-motion in depth. We found that angular viewpoint oscillation systematically reduced the onset latencies and increased the overall strength of vection in depth, compared with pure radial flow. In Experiment 2, we compared vection strength between radial flow displays with either added angular oscillation or continuous spiral rotation of equivalent peak velocity around the viewing axis (62.8°/s), and found that angular viewpoint oscillation generated the strongest vection. In Experiment 3, we found that pure radial flow with or without continuous spiral rotation produced radial motion aftereffects that lasted longer than that produced by radial flow with angular viewpoint oscillation. These findings support the view that the way viewpoint oscillation increases vection does not critically depend on motion parallax, but rather, on a changing pattern of retinal motion that serves to reduce visual adaptation and sustain sensitivity to optic flow.

Using virtual reality to augment perception, enhance sensorimotor adaptation, and change our minds

Technological advances that involve human sensorimotor processes can have both intended and unintended effects on the central nervous system (CNS). This mini review focuses on the use of virtual environments (VE) to augment brain functions by enhancing perception, eliciting automatic motor behavior, and inducing sensorimotor adaptation. VE technology is becoming increasingly prevalent in medical rehabilitation, training simulators, gaming, and entertainment. Although these VE applications have often been shown to optimize outcomes, whether it be to speed recovery, reduce training time, or enhance immersion and enjoyment, there are inherent drawbacks to environments that can potentially change sensorimotor calibration. Across numerous VE studies over the years, we have investigated the effects of combining visual and physical motion on perception, motor control, and adaptation. Recent results from our research involving exposure to dynamic passive motion within a visually-depicted VE reveal that short-term exposure to augmented sensorimotor discordance can result in systematic aftereffects that last beyond the exposure period. Whether these adaptations are advantageous or not, remains to be seen. Benefits as well as risks of using VE-driven sensorimotor stimulation to enhance brain processes will be discussed.

I speak fast when I move fast: the speed of illusory self-motion (vection) modulates the speed of utterances

Speed of utterance is an important factor in smooth and efficient conversation. We report a technique to increase utterance speed and that might improve a speaker's impression and information efficiency in conversation. We used a visual display consisting of optic flows in a large visual field that induced participants' illusory self-motion perception (vection). The speed of vection corresponded to the speed of the optic flows. Using this method, we investigated whether vection speed affects utterance speed. We presented fast- and slow-moving optic flow stimuli while dynamically swapping random dots presented to participants, during which time the participants were asked to talk for 2 min. Results revealed that the utterance speed was significantly faster in the fast optic flow condition. Our method could be a stepping stone for establishing a technique of modulating speech speed effectively.

Visual rotation axis and body position relative to the gravitational direction: Effects on circular vection

The visual–vestibular conflict theory asserts that visual–vestibular conflicts reduce vection and that vection strength is reduced with an increasing discrepancy between actual and expected vestibular activity. Most studies support this theory, although researchers have not always accepted them. To ascertain the conditions under which the theory of the visual–vestibular conflict can be applied, we measured circular vection strength accompanied by manipulation of the visual–otolith conflict by setting the axes of visual global motion (pitch, roll, and yaw) as either earth-horizontal or earth-vertical, using three different body positions (supine, left-lateral recumbent, and sitting upright). When the smaller stimulus was used, roll vection strength was greater with the visual–otolith conflict than without it, which contradicts the visual–vestibular conflict theory. We confirmed this result, as observers were able to distinguish circular vection from an illusory body tilt. Moreover, with observers in an upright position, the strength of yaw vection, which does not involve the visual–otolith conflict, increased and was almost equal to that of roll vection, which involves the visual–otolith conflict. This suggests that if the visual stimulus covers the entire visual field, the strength of circular vection around the earth-vertical axis exceeds that around the earth-horizontal axis, which is a finding consistent with the visual–vestibular conflict theory.

Directionless vection: A new illusory self-motion perception

We report a new visual illusion, “directionless vection.” When expanding and contracting optic flows are simultaneously presented in the same depth plane, observers can perceive illusory self-motion (vection) without direction.