Multiple channels of visual time perception

The neural link between stimulus duration and spatial location in the human visual hierarchy

Integrating spatial and temporal information is essential for our sensory experience. While psychophysical evidence suggests spatial dependencies in duration perception, few studies have directly tested the neural link between temporal and spatial processing. Using ultra-high-field functional MRI and neuronal-based modeling, we investigated how and where the processing and the representation of a visual stimulus duration is linked to that of its spatial location. Our results show a transition in duration coding: from monotonic and spatially-dependent in early visual cortex to unimodal and spatially-invariant in frontal cortex. Along the dorsal visual stream, particularly in the intraparietal sulcus (IPS), neuronal populations show common selective responses to both spatial and temporal information. In the IPS, spatial and temporal topographic organizations are also linked, although duration maps are smaller, less clustered, and more variable across participants. These findings help identify the mechanisms underlying human perception of visual duration and characterize the functional link between time and space processing, highlighting the importance of their interactions in shaping brain responses.

Memorability shapes perceived time (and vice versa)

Visual stimuli are known to vary in their perceived duration. Some visual stimuli are also known to linger for longer in memory. Yet, whether these two features of visual processing are linked is unknown. Despite early assumptions that time is an extracted or higher-order feature of perception, more recent work over the past two decades has demonstrated that timing may be instantiated within sensory modality circuits. A primary location for many of these studies is the visual system, where duration-sensitive responses have been demonstrated. Furthermore, visual stimulus features have been observed to shift perceived duration. These findings suggest that visual circuits mediate or construct perceived time. Here we present evidence across a series of experiments that perceived time is affected by the image properties of scene size, clutter and memorability. More specifically, we observe that scene size and memorability dilate time, whereas clutter contracts it. Furthermore, the durations of more memorable images are also perceived more precisely. Conversely, the longer the perceived duration of an image, the more memorable it is. To explain these findings, we applied a recurrent convolutional neural network model of the ventral visual system, in which images are progressively processed over time. We find that more memorable images are processed faster, and that this increase in processing speed predicts both the lengthening and the increased precision of perceived durations. These findings provide evidence for a link between image features, time perception and memory that can be further explored with models of visual processing.

Measuring the perception and metacognition of time

The ability of humans to identify and reproduce short time intervals (in the region of a second) may be affected by many factors ranging from the gender and personality of the individual observer, through the attentional state, to the precise spatiotemporal structure of the stimulus. The relative roles of these very different factors are a challenge to describe and define; several methodological approaches have been used to achieve this to varying degrees of success. Here we describe and model the results of a paradigm affording not only a first-order measurement of the perceived duration of an interval but also a second-order metacognitive judgement of perceived time. This approach, we argue, expands the form of the data generally collected in duration-judgements and allows more detailed comparison of psychophysical behavior to the underlying theory. We also describe a hierarchical Bayesian measurement model that performs a quantitative analysis of the trial-by-trial data calculating the variability of the temporal estimates and the metacognitive judgments allowing direct comparison between an actual and an ideal observer. We fit the model to data collected for judgements of 750 ms (bisecting 1500 ms) and 1500 ms (bisecting 3000 ms) intervals across three stimulus modalities (visual, audio, and audiovisual). This enhanced form of data on a given interval judgement and the ability to track its progression on a trial-by-trial basis offers a way of looking at the different roles that subject-based, task-based and stimulus-based factors have on the perception of time.

A metacognitive approach to the study of motion-induced duration biases reveals inter-individual differences in forming confidence judgments

Our ability to estimate the duration of subsecond visual events is prone to distortions, which depend on both sensory and decisional factors. To disambiguate between these two influences, we can look at the alignment between discrimination estimates of duration at the point of subjective equality and confidence estimates when the confidence about decisions is minimal, because observers should be maximally uncertain when two stimuli are perceptually the same. Here, we used this approach to investigate the relationship between the speed of a visual stimulus and its perceived duration. Participants were required to compare two intervals, report which had the longer duration, and then rate their confidence in that judgment. One of the intervals contained a stimulus drifting at a constant speed, whereas the stimulus embedded in the other interval could be stationary, linearly accelerating or decelerating, or drifting at the same speed. Discrimination estimates revealed duration compression for the stationary stimuli and, to a lesser degree, for the accelerating and decelerating stimuli. Confidence showed a similar pattern, but, overall, the confidence estimates were shifted more toward higher durations, pointing to a small contribution of decisional processes. A simple observer model, which assumes that both judgments are based on the same sensory information, captured well inter-individual differences in the criterion used to form a confidence judgment.

Duration judgments are mediated by the similarity with the temporal context

When we try to assess the duration of an event, we are often affected by external information. Studies on multiple timing have found that simultaneous timing information can produce an averaging or central tendency effect, where the perceived duration of the elements tends to be biased towards a general average. We wanted to assess how this effect induced by simultaneous distractors could depend on the temporal similarity between stimuli. We used a duration judgment task in which participants (n = 22) had to compare the duration of two identical targets (1 s) accompanied by simultaneous distractors of different durations (0.3, 0.7, 1.5 or 3 s). We found a central tendency effect, where duration judgments of the target were systematically biased towards the duration of the distractors that accompanied them. We put forward a model based on the concept of duration-channels that can explain the central tendency effect with only one estimated parameter. This parameter modulates the rate of decay of this effect as distractors duration become more different than the duration of the target.

Temporal perturbations cause movement-context independent but modality specific sensorimotor adaptation

Complex, goal-directed and time-critical movements require the processing of temporal features in sensory information as well as the fine-tuned temporal interplay of several effectors. Temporal estimates used to produce such behavior may thus be obtained through perceptual or motor processes. To disentangle the two options, we tested whether adaptation to a temporal perturbation in an interval reproduction task transfers to interval reproduction tasks with varying sensory information (visual appearance of targets, modality, and virtual reality [VR] environment or real-world) or varying movement types (continuous arm movements or brief clicking movements). Halfway through the experiments we introduced a temporal perturbation, such that continuous pointing movements were artificially slowed down in VR, causing participants to adapt their behavior to sustain performance. In four experiments, we found that sensorimotor adaptation to temporal perturbations is independent of environment context and movement type, but modality specific. Our findings suggest that motor errors induced by temporal sensorimotor adaptation affect the modality specific perceptual processing of temporal estimates.

Impact of delayed response on wearable cognitive assistance

Wearable cognitive assistants (WCA) are anticipated to become a widely-used application class, in conjunction with emerging network infrastructures like 5G that incorporate edge computing capabilities. While prototypical studies of such applications exist today, the relationship between infrastructure service provisioning and its implication for WCA usability is largely unexplored despite the relevance that these applications have for future networks. This paper presents an experimental study assessing how WCA users react to varying end-to-end delays induced by the application pipeline or infrastructure. Participants interacted directly with an instrumented task-guidance WCA as delays were introduced into the system in a controllable fashion. System and task state were tracked in real time, and biometric data from wearable sensors on the participants were recorded. Our results show that periods of extended system delay cause users to correspondingly (and substantially) slow down in their guided task execution, an effect that persists for a time after the system returns to a more responsive state. Furthermore, the slow-down in task execution is correlated with a personality trait, neuroticism, associated with intolerance for time delays. We show that our results implicate impaired cognitive planning, as contrasted with resource depletion or emotional arousal, as the reason for slowed user task executions under system delay. The findings have several implications for the design and operation of WCA applications as well as computational and communication infrastructure, and additionally for the development of performance analysis tools for WCA.

Differences in perceived durations between plausible biological and non-biological stimuli

Visual motion stimuli can sometimes distort our perception of time. This effect is dependent on the apparent speed of the moving stimulus, where faster stimuli are usually perceived lasting longer than slower stimuli. Although it has been shown that neural and cognitive processing of biological motion stimuli differ from non-biological motion stimuli, no study has yet investigated whether perceived durations of biological stimuli differ from non-biological stimuli across different speeds. Here, a prospective temporal reproduction task was used to assess that question. Biological motion stimuli consisted of a human silhouette running in place. Non-biological motion stimuli consisted of a rectangle moving in a pendular way. Amount and plausibility of movement for each stimulus and frame-rate (speed) were evaluated by an independent group of participants. Although the amount of movement perceived was positively correlated to frame rate both for biological and non-biological stimuli, movie clips involving biological motion stimuli were judged to last longer than non-biological motion stimuli only at frame rates for which movement was rated as plausible. These results suggest that plausible representations of biomechanical movement induce additional temporal distortions to those modulated by increases in stimulus speed. Moreover, most studies reporting neural and cognitive differences in the processing of biological and non-biological motion stimuli acquired neurophysiological data using fMRI. Here, we report differences in the processing of biological and non-biological motion stimuli across different speeds using functional near-infrared spectroscopy (fNIRS), a less costly and portable form of neurophysiological data acquisition.

A Network of Topographic Maps in Human Association Cortex Hierarchically Transforms Visual Timing-Selective Responses

Accurately timing sub-second sensory events is crucial when perceiving our dynamic world. This ability allows complex human behaviors that require timing-dependent multisensory integration and action planning. Such behaviors include perception and performance of speech, music, driving, and many sports. How are responses to sensory event timing processed for multisensory integration and action planning? We measured responses to viewing systematically changing visual event timing using ultra-high-field fMRI. We analyzed these responses with neural population response models selective for event duration and frequency, following behavioral, computational, and macaque action planning results and comparisons to alternative models. We found systematic local changes in timing preferences (recently described in supplementary motor area) in an extensive network of topographic timing maps, mirroring sensory cortices and other quantity processing networks. These timing maps were partially left lateralized and widely spread, from occipital visual areas through parietal multisensory areas to frontal action planning areas. Responses to event duration and frequency were closely linked. As in sensory cortical maps, response precision varied systematically with timing preferences, and timing selectivity systematically varied between maps. Progressing from posterior to anterior maps, responses to multiple events were increasingly integrated, response selectivity narrowed, and responses focused increasingly on the middle of the presented timing range. These timing maps largely overlap with numerosity and visual field map networks. In both visual timing map and visual field map networks, selective responses and topographic map organization may facilitate hierarchical transformations by allowing neural populations to interact over minimal distances.

•

Many brain areas show neural responses to specific ranges of visual event timing

•

Timing preferences change gradually in these areas, forming topographic timing maps

•

Neural response properties hierarchically transform from visual to premotor areas

•

Timing, numerosity, and visual field map networks are distinct but largely overlap

Short- and long-term forms of neural adaptation: An ERP investigation of dynamic motion aftereffects

Adaptation is essential to interact with a dynamic and changing environment, and can be observed on different timescales. Previous studies on a motion paradigm called dynamic motion aftereffect (dMAE) showed that neural adaptation can establish even in very short timescales. However, the neural mechanisms underlying such rapid form of neural plasticity is still debated. In the present study, short- and long-term forms of neural plasticity were investigated using dynamic motion aftereffect combined with EEG (Electroencephalogram). Participants were adapted to directional drifting gratings for either short (640 msec) or long (6.4 sec) durations. Both adaptation durations led to motion aftereffects on the perceived direction of a dynamic and directionally ambiguous test pattern, but the long adaptation produced stronger dMAE. In line with behavioral results, we found robust changes in the event-related potentials elicited by the dynamic test pattern within 64-112 msec time range. These changes were mainly clustered over occipital and parieto-occipital scalp sites. Within this time range, the aftereffects induced by long adaptation were stronger than those by short adaptation. Moreover, the aftereffects by each adaptation duration were in the opposite direction. Overall, these EEG findings suggest that dMAEs reflect changes in cortical areas mediating low- and mid-level visual motion processing. They further provide evidence that short- and long-term forms of motion adaptation lead to distinct changes in neural activity, and hence support the view that adaptation is an active time-dependent process which involves different neural mechanisms.

Short-term global motion adaptation induces a compression in the subjective duration of dynamic visual events

Apparent duration can be manipulated in a local region of visual field by long-term adaptation to motion or flicker (Johnston, Arnold, & Nishida, 2006). These effects show narrow spatial tuning (Ayhan, Bruno, Nishida, & Johnston, 2009), as well as retinotopic position dependency (Bruno, Ayhan, & Johnston, 2010), supporting early locus in the visual pathway. Here, we introduce a new effect using RDK as a short-term visual adaptor and demonstrate that a brief, subsecond range adaptation induces a significant subjective duration compression (∼10%) on a subsequently presented test stimulus (RDK pattern) only for global motion patterns drifting at 50% motion coherence but not for those drifting at 0% coherence, suggesting a higher level area as a source of origin. In another set of experiments using a plaid stimulus as the adaptor and gratings as the tests, we report again a significant duration compression following a brief motion adaptation, although the effect does not seem to be consistently selective for a particular direction of the standard test relative to that of the plaid adaptor (two-dimensional motion) or its components (one-dimensional motion). Finally, we conduct an experiment using shutter glasses and find that the effects of a short-term adaptor presented monocularly to one eye transfer to the nonadapted eye, providing evidence for the interocular transfer. In a series of control experiments, we also show that the duration effects cannot be explained by adaptation-induced changes in perceived speed, perceived onset-and-offset, and attentional resource allocation. Overall, the duration compression effect requiring motion coherence in RDK, persisting in plaid stimulus, and showing interocular transfer imply explicit genuine mechanisms mediating duration effects in the higher level motion areas.

Adaptation reveals multi-stage coding of visual duration

In conflict with historically dominant models of time perception, recent evidence suggests that the encoding of our environment's temporal properties may not require a separate class of neurons whose raison d'être is the dedicated processing of temporal information. If true, it follows that temporal processing should be imbued with the known selectivity found within non-temporal neurons. In the current study, we tested this hypothesis for the processing of a poorly understood stimulus parameter: visual event duration. We used sensory adaptation techniques to generate duration aftereffects: bidirectional distortions of perceived duration. Presenting adapting and test durations to the same vs different eyes utilises the visual system's anatomical progression from monocular, pre-cortical neurons to their binocular, cortical counterparts. Duration aftereffects exhibited robust inter-ocular transfer alongside a small but significant contribution from monocular mechanisms. We then used novel stimuli which provided duration information that was invisible to monocular neurons. These stimuli generated robust duration aftereffects which showed partial selectivity for adapt-test changes in retinal disparity. Our findings reveal distinct duration encoding mechanisms at monocular, depth-selective and depth-invariant stages of the visual hierarchy.

Dilation and Constriction of Subjective Time Based on Observed Walking Speed

The physical properties of events are known to modulate perceived time. This study tested the effect of different quantitative (walking speed) and qualitative (walking-forward vs. walking-backward) features of observed motion on time perception in three complementary experiments. Participants were tested in the temporal discrimination (bisection) task, in which they were asked to categorize durations of walking animations as "short" or "long." We predicted the faster observed walking to speed up temporal integration and thereby to shift the point of subjective equality leftward, and this effect to increase monotonically with increasing walking speed. To this end, we tested participants with two different ranges of walking speeds in Experiment 1 and 2 and observed a parametric effect of walking speed on perceived time irrespective of the direction of walking (forward vs. rewound forward walking). Experiment 3 contained a more plausible backward walking animation compared to the rewound walking animation used in Experiments 1 and 2 (as validated based on independent subjective ratings). The effect of walking-speed and the lack of the effect of walking direction on perceived time were replicated in Experiment 3. Our results suggest a strong link between the speed but not the direction of perceived biological motion and subjective time.

Rhythmic motor behaviour influences perception of visual time

Temporal processing is fundamental for an accurate synchronization between motor behaviour and sensory processing. Here, we investigate how motor timing during rhythmic tapping influences perception of visual time. Participants listen to a sequence of four auditory tones played at 1 Hz and continue the sequence (without auditory stimulation) by tapping four times with their finger. During finger tapping, they are presented with an empty visual interval and are asked to judge its length compared to a previously internalized interval of 150 ms. The visual temporal estimates show non-monotonic changes locked to the finger tapping: perceived time is maximally expanded at halftime between the two consecutive finger taps, and maximally compressed near tap onsets. Importantly, the temporal dynamics of the perceptual time distortion scales linearly with the timing of the motor tapping, with maximal expansion always being anchored to the centre of the inter-tap interval. These results reveal an intrinsic coupling between distortion of perceptual time and production of self-timed motor rhythms, suggesting the existence of a timing mechanism that keeps perception and action accurately synchronized.

Motion-induced compression of perceived numerosity

It has been recently proposed that space, time, and number might share a common representation in the brain. Evidence supporting this idea comes from adaptation studies demonstrating that prolonged exposure to a given stimulus feature distorts the perception of different characteristics. For example, visual motion adaptation affects both perceived position and duration of subsequent stimuli presented in the adapted location. Here, we tested whether motion adaptation also affects perceived numerosity, by testing the effect of adaptation to translating or rotating stimuli moving either at high (20 Hz) or low (5 Hz) speed. Adaptation to fast translational motion yielded a robust reduction in the apparent numerosity of the adapted stimulus (~25%) while adaptation to slow translational or circular motion (either 20 Hz or 5 Hz) yielded a weaker but still significant compression. Control experiments suggested that none of these results could be accounted for in terms of stimulus masking. Taken together, our results are consistent with the extant literature supporting the idea of a generalized magnitude system underlying the representation of numerosity, space and time via common metrics. However, as changes in perceived numerosity co-varied with both adapting motion profile and speed, our evidence also suggests complex and asymmetric interactions between different magnitude representations.

Individual differences in first- and second-order temporal judgment

The ability of subjects to identify and reproduce brief temporal intervals is influenced by many factors whether they be stimulus-based, task-based or subject-based. The current study examines the role individual differences play in subsecond and suprasecond timing judgments, using the schizoptypy personality scale as a test-case approach for quantifying a broad range of individual differences. In two experiments, 129 (Experiment 1) and 141 (Experiment 2) subjects completed the O-LIFE personality questionnaire prior to performing a modified temporal-bisection task. In the bisection task, subjects responded to two identical instantiations of a luminance grating presented in a 4deg window, 4deg above fixation for 1.5 s (Experiment 1) or 3 s (Experiment 2). Subjects initiated presentation with a button-press, and released the button when they considered the stimulus to be half-way through (750/1500 ms). Subjects were then asked to indicate their 'most accurate estimate' of the two intervals. In this way we measure both performance on the task (a first-order measure) and the subjects' knowledge of their performance (a second-order measure). In Experiment 1 the effect of grating-drift and feedback on performance was also examined. Experiment 2 focused on the static/no-feedback condition. For the group data, Experiment 1 showed a significant effect of presentation order in the baseline condition (no feedback), which disappeared when feedback was provided. Moving the stimulus had no effect on perceived duration. Experiment 2 showed no effect of stimulus presentation order. This elimination of the subsecond order-effect was at the expense of accuracy, as the mid-point of the suprasecond interval was generally underestimated. Response precision increased as a proportion of total duration, reducing the variance below that predicted by Weber's law. This result is consistent with a breakdown of the scalar properties of time perception in the early suprasecond range. All subjects showed good insight into their own performance, though that insight did not necessarily correlate with the veridical bisection point. In terms of personality, we found evidence of significant differences in performance along the Unusual Experiences subscale, of most theoretical interest here, in the subsecond condition only. There was also significant correlation with Impulsive Nonconformity and Cognitive Disorganisation in the sub- and suprasecond conditions, respectively. Overall, these data support a partial dissociation of timing mechanisms at very short and slightly longer intervals. Further, these results suggest that perception is not the only critical mitigator of confidence in temporal experience, since individuals can effectively compensate for differences in perception at the level of metacognition in early suprasecond time. Though there are individual differences in performance, these are perhaps less than expected from previous reports and indicate an effective timing mechanism dealing with brief durations independent of the influence of significant personality trait differences.

Rate after-effects fail to transfer cross-modally: Evidence for distributed sensory timing mechanisms

Accurate time perception is critical for a number of human behaviours, such as understanding speech and the appreciation of music. However, it remains unresolved whether sensory time perception is mediated by a central timing component regulating all senses, or by a set of distributed mechanisms, each dedicated to a single sensory modality and operating in a largely independent manner. To address this issue, we conducted a range of unimodal and cross-modal rate adaptation experiments, in order to establish the degree of specificity of classical after-effects of sensory adaptation. Adapting to a fast rate of sensory stimulation typically makes a moderate rate appear slower (repulsive after-effect), and vice versa. A central timing hypothesis predicts general transfer of adaptation effects across modalities, whilst distributed mechanisms predict a high degree of sensory selectivity. Rate perception was quantified by a method of temporal reproduction across all combinations of visual, auditory and tactile senses. Robust repulsive after-effects were observed in all unimodal rate conditions, but were not observed for any cross-modal pairings. Our results show that sensory timing abilities are adaptable but, crucially, that this change is modality-specific - an outcome that is consistent with a distributed sensory timing hypothesis.

Relative Time Compression for Slow-Motion Stimuli through Rapid Recalibration

A number of psychophysical studies have shown that moving stimuli appear to last longer than static stimuli. Here, we report that the perceived duration for slow moving stimuli can be shorter than for static stimuli under specific circumstances. Observers were tested using natural movies presented at various speeds (0.0× = static, 0.25× = slow, or 1.9× = fast, relative to original speed) and indicated whether test duration was perceived as longer or shorter than comparison movies presented at their original speed. While fast movies were perceived as longer than slow and static movies (in accordance with previous studies), we found that slow movies were perceived as shorter (i.e., time compressed) compared to static images. Similar results were obtained for artificial stimuli consisting of drifting gratings. However, time compression for slow stimuli disappeared if comparison stimuli were replaced by a white static disk that removed repetitive exposures to moving stimuli. Results suggest that duration estimation is modulated by contextual effects induced by the specific diet – or distribution – of prior visual stimuli to which observers are exposed. A simple model, which includes a rapid recalibration of human time estimation via adaptation to preceding stimuli, succeeds in reproducing our experimental data.