Rapid Audiovisual Temporal Recalibration Generalises Across Spatial Location

No effect of spatial congruence on rapid temporal recalibration to audiovisual asynchrony

The brain integrates multisensory information to construct coherent perceptual representations based on spatial and temporal congruence. Intriguingly, multisensory timing perception can be flexibly calibrated. Repeated exposure to audiovisual asynchrony induces shifts in subjective simultaneity (temporal recalibration). Spatial congruence is known to serve as a grouping cue for recalibration when the audiovisual temporal relationship is ambiguous during exposure. A single exposure to audiovisual asynchrony can also trigger temporal recalibration (rapid recalibration). However, it has been suggested that the underlying mechanisms of these temporal recalibrations differ. Here, we examined whether spatial congruence can be a grouping cue for rapid recalibration when audiovisual pairs are not defined by temporal relationships. Participants made a simultaneity judgment for a pair of audiovisual stimuli after adapting three consecutive stimuli once in a "light-sound-light" or "sound-light-sound" order with an equal temporal interval. The spatial positions of the adapting stimuli were manipulated as an audiovisual pair from the same position (e.g., left) and the remaining stimulus from another position (e.g., right). In three experiments, the spatial congruence of the audiovisual adapting stimuli did not show a modulatory effect, while we replicated the rapid recalibration effects. Rather, rapid recalibration occurred according to the temporal order of the first light and sound. Our findings suggest that, in contrast to temporal recalibration with repeated exposure, the perceptual systems underlying rapid recalibration simply combine individual visual and auditory inputs based on the order in which they arrive.

Change of rapid temporal recalibration magnitude for audiovisual asynchrony with modulation of temporal binding window width: A preliminary investigation

The subjective synchrony perception for audiovisual stimuli is affected by previous temporal information. The point of subjective simultaneity is shifted toward the same asynchronous direction of audiovisual stimuli in a previous trial. This phenomenon is called "rapid temporal recalibration." The factors that modulate the magnitude of rapid temporal recalibration have not been fully investigated. Previously, a positive correlation has been found between the magnitude of rapid temporal recalibration and the width of the temporal binding window (TBW). This preliminary study examined the causal relationship between TBW size and rapid recalibration magnitude using a single experimental group comparison design. In this experiment, the magnitude of rapid recalibration was compared before and after perceptual training, which narrowed the TBW width. The results indicated that the magnitude of rapid recalibration was reduced by perceptual training. Therefore, it was speculated that TBW size determined the magnitude of rapid recalibration. This causal relationship helps elucidate the mechanisms of the adaptation for temporal lags between visual and auditory sensations.

Audiovisual illusion training improves multisensory temporal integration

When we perceive external physical stimuli from the environment, the brain must remain somewhat flexible to unaligned stimuli within a specific range, as multisensory signals are subject to different transmission and processing delays. Recent studies have shown that the width of the 'temporal binding window (TBW)' can be reduced by perceptual learning. However, to date, the vast majority of studies examining the mechanisms of perceptual learning have focused on experience-dependent effects, failing to reach a consensus on its relationship with the underlying perception influenced by audiovisual illusion. The sound-induced flash illusion (SiFI) training is a reliable function for improving perceptual sensitivity. The present study utilized the classic auditory-dominated SiFI paradigm with feedback training to investigate the effect of a 5-day SiFI training on multisensory temporal integration, as evaluated by a simultaneity judgment (SJ) task and temporal order judgment (TOJ) task. We demonstrate that audiovisual illusion training enhances multisensory temporal integration precision in the form of (i) the point of subjective simultaneity (PSS) shifts to reality (0 ms) and (ii) a narrowing TBW. The results are consistent with a Bayesian model of causal inference, suggesting that perception learning reduce the susceptibility to SiFI, whilst improving the precision of audiovisual temporal estimation.

The development of audio-visual temporal precision precedes its rapid recalibration

Through development, multisensory systems reach a balance between stability and flexibility: the systems integrate optimally cross-modal signals from the same events, while remaining adaptive to environmental changes. Is continuous intersensory recalibration required to shape optimal integration mechanisms, or does multisensory integration develop prior to recalibration? Here, we examined the development of multisensory integration and rapid recalibration in the temporal domain by re-analyzing published datasets for audio-visual, audio-tactile, and visual-tactile combinations. Results showed that children reach an adult level of precision in audio-visual simultaneity perception and show the first sign of rapid recalibration at 9 years of age. In contrast, there was very weak rapid recalibration for other cross-modal combinations at all ages, even when adult levels of temporal precision had developed. Thus, the development of audio-visual rapid recalibration appears to require the maturation of temporal precision. It may serve to accommodate distance-dependent travel time differences between light and sound.

Visual field differences in temporal synchrony processing for audio-visual stimuli

Audio-visual integration relies on temporal synchrony between visual and auditory inputs. However, differences in traveling and transmitting speeds between visual and auditory stimuli exist; therefore, audio-visual synchrony perception exhibits flexible functions. The processing speed of visual stimuli affects the perception of audio-visual synchrony. The present study examined the effects of visual fields, in which visual stimuli are presented, for the processing of audio-visual temporal synchrony. The point of subjective simultaneity, the temporal binding window, and the rapid recalibration effect were measured using temporal order judgment, simultaneity judgment, and stream/bounce perception, because different mechanisms of temporal processing have been suggested among these three paradigms. The results indicate that auditory stimuli should be presented earlier for visual stimuli in the central visual field than in the peripheral visual field condition in order to perceive subjective simultaneity in the temporal order judgment task conducted in this study. Meanwhile, the subjective simultaneity bandwidth was broader in the central visual field than in the peripheral visual field during the simultaneity judgment task. In the stream/bounce perception task, neither the point of subjective simultaneity nor the temporal binding window differed between the two types of visual fields. Moreover, rapid recalibration occurred in both visual fields during the simultaneity judgment tasks. However, during the temporal order judgment task and stream/bounce perception, rapid recalibration occurred only in the central visual field. These results suggest that differences in visual processing speed based on the visual field modulate the temporal processing of audio-visual stimuli. Furthermore, these three tasks, temporal order judgment, simultaneity judgment, and stream/bounce perception, each have distinct functional characteristics for audio-visual synchrony perception. Future studies are necessary to confirm the effects of compensation regarding differences in the temporal resolution of the visual field in later cortical visual pathways on visual field differences in audio-visual temporal synchrony.

Rapid Temporal Recalibration to Audiovisual Asynchrony Occurs Across the Difference in Neural Processing Speed Based on Spatial Frequency

Audiovisual integration relies on temporal synchrony between visual and auditory stimuli. The brain rapidly adapts to audiovisual asynchronous events by shifting the timing of subjective synchrony in the direction of the leading modality of the most recent event, a process called rapid temporal recalibration. This phenomenon is the flexible function of audiovisual synchrony perception. Previous studies found that neural processing speed based on spatial frequency (SF) affects the timing of subjective synchrony. This study examined the effects of SF on the rapid temporal recalibration process by discriminating whether the presentation of the visual and auditory stimuli was simultaneous. I compared the magnitudes of the recalibration effect between low and high SF visual stimuli using two techniques. First, I randomly presented each SF accompanied by a tone during one session, then in a second experiment, only a single SF was paired with the tone throughout the one session. The results indicated that rapid recalibration occurred regardless of difference in presented SF between preceding and test trials. The recalibration magnitude did not significantly differ between the SF conditions. These findings confirm that intersensory temporal process is important to produce rapid recalibration and suggest that rapid recalibration can be induced by the simultaneity judgment criterion changes attributed to the low-level temporal information of audiovisual events.