A Roving Dual-Presentation Simultaneity-Judgment Task to Estimate the Point of Subjective Simultaneity

Olfactory-gustatory simultaneity judgments: A preliminary study on the congruency-dependent temporal window of multisensory binding

BACKGROUND

A greater congruency of audio and video expands the temporal binding window (TBW). A similar phenomenon may occur with a combination of odor and taste, which are the main components of flavor.

OBJECTIVE

TBW is defined as the temporal resolution of synchrony perception. The larger the TBW, the lower the resolution. We hypothesized that the more congruent the odor and taste, the lower the temporal resolution of synchrony perception.

METHODS

To examine this hypothesis, 10 female participants performed simultaneity judgment (SJ) tasks under congruent (soy sauce odor with saline) and incongruent (cherry tree leaf odor with saline) conditions and evaluated the congruency with saltiness for the two odors. In the SJ tasks, participants reported whether odor and taste were presented simultaneously or successively. We assumed a Gaussian distribution for the temporal distributions of the simultaneous response rates and calculated the approximations. In addition, we computed the half width at half height (HWHH) as an index of TBW based on the coefficient of approximation for the temporal distribution of the simultaneous response rates.

RESULTS

HWHH was significantly larger under congruent condition than under incongruent condition. In addition, congruency with saltiness had a significant moderate positive correlation with HWHH.

CONCLUSION

The larger the HWHH, the lower the temporal resolution of synchrony perception, supporting the hypothesis. This study suggests that the width of TBW depends on the cross-modal congruency similar to the case for audio-visual SJs. However, methodological improvements, including a larger sample size and gender-independent recruitment of participants, are essential to enhance the reliability of the findings because some of the results did not provide sufficient ESs or power.

Neural-latency noise places limits on human sensitivity to the timing of events

The brain-time account posits that the physical timing of sensory-evoked neural activity determines the perceived timing of corresponding sensory events. A canonical model formalises this account for tasks such as simultaneity and order judgements: Signals arrive at a decision centre in an order, and at a temporal offset, shaped by neural propagation times. This model assumes that the noise affecting people's temporal judgements is primarily neural-latency noise, i.e. variation in propagation times across trials, but this assumption has received little scrutiny. Here, we recorded EEG alongside simultaneity judgements from 50 participants in response to combinations of visual, auditory and tactile stimuli. Bootstrapping of ERP components was used to estimate neural-latency noise, and simultaneity judgements were modelled to estimate the precision of timing judgements. We obtained the predicted correlation between neural and behavioural measures of latency noise, supporting a fundamental feature of the canonical model of perceived timing.

The simultaneous oddball: Oddball presentation does not affect simultaneity judgments

The oddball duration illusion describes how a rare or nonrepeated stimulus is perceived as lasting longer than a common or repeated stimulus. It has been argued that the oddball duration illusion could emerge because of an earlier perceived onset of an oddball stimulus. However, most methods used to assess the perceived duration of an oddball stimulus are ill suited to detect onset effects. Therefore, in the current article, I tested the perceived onset of oddball and standard stimuli using a simultaneity judgment task. In Experiments 1 and 2, repetition and rarity of the target stimulus were varied, and participants were required to judge whether the target stimulus and another stimulus were concurrent. In Experiment 3, I tested whether a brief initial stimulus could act as a conditioning stimulus in the oddball duration illusion. This was to ensure an oddball duration illusion could have occurred given the short duration of stimuli in the first two experiments. In both the first two experiments, I found moderate support for no onset-based difference between oddball and nonoddball stimuli. In Experiment 3, I found that a short conditioning stimulus could still lead to the oddball duration illusion occurring, removing this possible explanation for the null result. Experiment 4 showed that an oddball duration illusion could emerge given the rarity of the stimulus and a concurrent sound. In sum, the current article found evidence against an onset-based explanation of the oddball duration illusion.

Audio-Visual Training in Older Adults: 2-Interval-Forced Choice Task Improves Performance

A growing interest in ameliorating multisensory perception deficits in older adults arises from recent evidence showing that impaired multisensory processing, particularly in the temporal domain, may be associated with cognitive and functional impairments. Perceptual training has proved successful in improving multisensory temporal processing in young adults, but few studies have investigated this training approach in older adults. In the present study we used a simultaneity (or synchronicity) judgement task with feedback, to train the audio-visual abilities of community-dwelling, cognitively healthy older adults. We recruited 23 older adults (M = 74.17, SD = 6.23) and a group of 20 young adults (M = 24.20, SD = 4.23) who served as a comparison. Participants were tested before and after perceptual training using a 2-Interval Forced Choice Task (2-IFC); and the Sound-Induced Flash Illusion (SIFI). After 3 days of training, participants improved on the 2-IFC task, with a significant narrowing of the temporal window of integration (TWI) found for both groups. Generalization of training effects was not found, with no post-training differences in perceptual sensitivity to the SIFI for either group. These findings provide evidence perceptual narrowing can be achieved in older as well as younger adults after 3 days of perceptual training. These results provide useful information for future studies attempting to improve audio-visual temporal discrimination abilities in older people.

Audiovisual temporal integration: Cognitive processing, neural mechanisms, developmental trajectory and potential interventions

To integrate auditory and visual signals into a unified percept, the paired stimuli must co-occur within a limited time window known as the Temporal Binding Window (TBW). The width of the TBW, a proxy of audiovisual temporal integration ability, has been found to be correlated with higher-order cognitive and social functions. A comprehensive review of studies investigating audiovisual TBW reveals several findings: (1) a wide range of top-down processes and bottom-up features can modulate the width of the TBW, facilitating adaptation to the changing and multisensory external environment; (2) a large-scale brain network works in coordination to ensure successful detection of audiovisual (a)synchrony; (3) developmentally, audiovisual TBW follows a U-shaped pattern across the lifespan, with a protracted developmental course into late adolescence and rebounding in size again in late life; (4) an enlarged TBW is characteristic of a number of neurodevelopmental disorders; and (5) the TBW is highly flexible via perceptual and musical training. Interventions targeting the TBW may be able to improve multisensory function and ameliorate social communicative symptoms in clinical populations.

Aging Impairs Temporal Sensitivity, but not Perceptual Synchrony, Across Modalities

Encoding the temporal properties of external signals that comprise multimodal events is a major factor guiding everyday experience. However, during the natural aging process, impairments to sensory processing can profoundly affect multimodal temporal perception. Various mechanisms can contribute to temporal perception, and thus it is imperative to understand how each can be affected by age. In the current study, using three different temporal order judgement tasks (unisensory, multisensory, and sensorimotor), we investigated the effects of age on two separate temporal processes: synchronization and integration of multiple signals. These two processes rely on different aspects of temporal information, either the temporal alignment of processed signals or the integration/segregation of signals arising from different modalities, respectively. Results showed that the ability to integrate/segregate multiple signals decreased with age regardless of the task, and that the magnitude of such impairment correlated across tasks, suggesting a widespread mechanism affected by age. In contrast, perceptual synchrony remained stable with age, revealing a distinct intact mechanism. Overall, results from this study suggest that aging has differential effects on temporal processing, and general impairments with aging may impact global temporal sensitivity while context-dependent processes remain unaffected.

Simultaneity and Temporal Order Judgments Are Coded Differently and Change With Age: An Event-Related Potential Study

Multisensory integration is required for a number of daily living tasks where the inability to accurately identify simultaneity and temporality of multisensory events results in errors in judgment leading to poor decision-making and dangerous behavior. Previously, our lab discovered that older adults exhibited impaired timing of audiovisual events, particularly when making temporal order judgments (TOJs). Simultaneity judgments (SJs), however, were preserved across the lifespan. Here, we investigate the difference between the TOJ and SJ tasks in younger and older adults to assess neural processing differences between these two tasks and across the lifespan. Event-related potentials (ERPs) were studied to determine between-task and between-age differences. Results revealed task specific differences in perceiving simultaneity and temporal order, suggesting that each task may be subserved via different neural mechanisms. Here, auditory N1 and visual P1 ERP amplitudes confirmed that unisensory processing of audiovisual stimuli did not differ between the two tasks within both younger and older groups, indicating that performance differences between tasks arise either from multisensory integration or higher-level decision-making. Compared to younger adults, older adults showed a sustained higher auditory N1 ERP amplitude response across SOAs, suggestive of broader response properties from an extended temporal binding window. Our work provides compelling evidence that different neural mechanisms subserve the SJ and TOJ tasks and that simultaneity and temporal order perception are coded differently and change with age.

Sight and sound persistently out of synch: stable individual differences in audiovisual synchronisation revealed by implicit measures of lip-voice integration

Are sight and sound out of synch? Signs that they are have been dismissed for over two centuries as an artefact of attentional and response bias, to which traditional subjective methods are prone. To avoid such biases, we measured performance on objective tasks that depend implicitly on achieving good lip-synch. We measured the McGurk effect (in which incongruent lip-voice pairs evoke illusory phonemes), and also identification of degraded speech, while manipulating audiovisual asynchrony. Peak performance was found at an average auditory lag of ~100 ms, but this varied widely between individuals. Participants’ individual optimal asynchronies showed trait-like stability when the same task was re-tested one week later, but measures based on different tasks did not correlate. This discounts the possible influence of common biasing factors, suggesting instead that our different tasks probe different brain networks, each subject to their own intrinsic auditory and visual processing latencies. Our findings call for renewed interest in the biological causes and cognitive consequences of individual sensory asynchronies, leading potentially to fresh insights into the neural representation of sensory timing. A concrete implication is that speech comprehension might be enhanced, by first measuring each individual’s optimal asynchrony and then applying a compensatory auditory delay.

A tilt after-effect for images of buildings: evidence of selectivity for the orientation of everyday scenes

The tilt after-effect (TAE) is thought to be a manifestation of gain control in mechanisms selective for spatial orientation in visual stimuli. It has been demonstrated with luminance-defined stripes, contrast-defined stripes, orientation-defined stripes and even with natural images. Of course, all images can be decomposed into a sum of stripes, so it should not be surprising to find a TAE when adapting and test images contain stripes that differ by 15° or so. We show this latter condition is not necessary for the TAE with natural images: adaptation to slightly tilted and vertically filtered houses produced a 'repulsive' bias in the perceived orientation of horizontally filtered houses. These results suggest gain control in mechanisms selective for spatial orientation in natural images.